2023-10-06T11:08:58.591 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T11:08:58.591 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395 --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/orig.config.yaml 2023-10-06T11:08:58.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:08:58.720 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} email: yweinste@redhat.com first_in_suite: false job_id: '7412395' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 client snapdir: .customsnapfuse debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: snapdirname: .customsnapkernel thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T11:08:58.720 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T11:08:58.721 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T11:08:58.721 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T11:08:58.723 INFO:teuthology.task.internal:Checking packages... 2023-10-06T11:08:58.747 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T11:08:58.747 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T11:08:58.748 INFO:teuthology.packaging:ref: None 2023-10-06T11:08:58.748 INFO:teuthology.packaging:tag: None 2023-10-06T11:08:58.748 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:08:58.748 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:08:58.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:08:58.969 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:08:58.969 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T11:08:58.976 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T11:08:58.976 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T11:08:59.012 INFO:teuthology.task.internal:Saving configuration 2023-10-06T11:08:59.033 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T11:08:59.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:08:59.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:03:08.080798', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T11:08:59.089 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:03:08.082571', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T11:08:59.090 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T11:08:59.096 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T11:08:59.096 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T11:08:59.096 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T11:08:59.171 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffad6b6f040>, signals=[15]) 2023-10-06T11:08:59.171 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T11:08:59.178 INFO:teuthology.task.internal:Opening connections... 2023-10-06T11:08:59.178 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:08:59.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:08:59.230 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:08:59.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:08:59.281 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T11:08:59.287 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2023-10-06T11:08:59.311 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2023-10-06T11:08:59.311 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2023-10-06T11:08:59.361 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Ubuntu" 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:ID=ubuntu 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE=debian 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="20.04" 2023-10-06T11:08:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T11:08:59.363 INFO:teuthology.orchestra.run.smithi007.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T11:08:59.363 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T11:08:59.363 INFO:teuthology.orchestra.run.smithi007.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T11:08:59.363 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_CODENAME=focal 2023-10-06T11:08:59.363 INFO:teuthology.orchestra.run.smithi007.stdout:UBUNTU_CODENAME=focal 2023-10-06T11:08:59.364 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2023-10-06T11:08:59.388 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2023-10-06T11:08:59.395 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-10-06T11:08:59.395 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-10-06T11:08:59.443 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Ubuntu" 2023-10-06T11:08:59.443 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:ID=ubuntu 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE=debian 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="20.04" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_CODENAME=focal 2023-10-06T11:08:59.444 INFO:teuthology.orchestra.run.smithi050.stdout:UBUNTU_CODENAME=focal 2023-10-06T11:08:59.445 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-10-06T11:08:59.463 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T11:08:59.484 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T11:08:59.496 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T11:08:59.497 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:08:59.499 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:08:59.504 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T11:08:59.510 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T11:08:59.510 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:08:59.545 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:08:59.559 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T11:08:59.610 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T11:08:59.624 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T11:08:59.624 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T11:08:59.625 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:08:59.625 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:08:59.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:08:59.631 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-10-06T11:08:59.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:08:59.632 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-10-06T11:08:59.638 INFO:teuthology.orchestra.run.smithi050.stdout:5.4.0-124-generic 2023-10-06T11:08:59.638 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.4.0-124-generic 2023-10-06T11:08:59.638 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get clean 2023-10-06T11:08:59.677 INFO:teuthology.orchestra.run.smithi007.stdout:5.4.0-124-generic 2023-10-06T11:08:59.677 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.4.0-124-generic 2023-10-06T11:08:59.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2023-10-06T11:08:59.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2023-10-06T11:08:59.780 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-10-06T11:08:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:08:59.984 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-06T11:09:00.033 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-06T11:09:00.068 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-06T11:09:00.070 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:09:00.080 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-06T11:09:00.130 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-06T11:09:00.406 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-06T11:09:00.472 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-06T11:09:00.513 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-06T11:09:00.568 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-06T11:09:00.572 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-06T11:09:00.580 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-06T11:09:00.581 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-06T11:09:00.582 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-06T11:09:00.621 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-06T11:09:00.624 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T11:09:00.625 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-06T11:09:00.647 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-06T11:09:00.655 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-06T11:09:00.660 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-06T11:09:00.666 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-06T11:09:00.667 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-06T11:09:00.667 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-06T11:09:00.668 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-06T11:09:00.690 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-06T11:09:00.691 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-06T11:09:00.692 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-06T11:09:00.692 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-06T11:09:00.692 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-06T11:09:00.696 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-06T11:09:00.707 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-06T11:09:00.729 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-06T11:09:00.914 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-06T11:09:01.039 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-06T11:09:01.102 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-06T11:09:01.117 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-06T11:09:01.118 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-06T11:09:01.118 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-06T11:09:01.186 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-06T11:09:01.192 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T11:09:01.192 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-06T11:09:01.204 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-06T11:09:01.205 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-06T11:09:01.214 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-06T11:09:01.220 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-06T11:09:01.227 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-06T11:09:01.228 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-06T11:09:01.229 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-06T11:09:01.229 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-06T11:09:01.229 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-06T11:09:01.230 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-06T11:09:01.299 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-06T11:09:01.304 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-06T11:09:01.304 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T11:09:01.311 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T11:09:01.311 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-06T11:09:01.331 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-06T11:09:01.405 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-06T11:09:01.468 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-06T11:09:01.508 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-06T11:09:01.822 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-06T11:09:01.827 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-06T11:09:01.835 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-06T11:09:01.838 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-06T11:09:01.840 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-06T11:09:01.924 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-06T11:09:01.930 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-06T11:09:01.932 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-06T11:09:01.941 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-06T11:09:01.942 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-06T11:09:02.055 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-06T11:09:02.164 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-06T11:09:02.184 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-06T11:09:02.186 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-06T11:09:02.187 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-06T11:09:02.189 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-06T11:09:02.189 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-06T11:09:02.190 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-06T11:09:02.192 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-06T11:09:02.195 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-06T11:09:02.359 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-06T11:09:02.522 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-06T11:09:02.574 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-06T11:09:04.540 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 17.6 MB in 3s (6,442 kB/s) 2023-10-06T11:09:05.032 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 17.6 MB in 3s (5,606 kB/s) 2023-10-06T11:09:05.946 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:09:05.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T11:09:06.038 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:09:06.245 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2023-10-06T11:09:06.246 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2023-10-06T11:09:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2023-10-06T11:09:06.435 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T11:09:06.435 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-06T11:09:06.435 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-06T11:09:06.436 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2023-10-06T11:09:06.436 INFO:teuthology.orchestra.run.smithi007.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-06T11:09:06.436 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2023-10-06T11:09:06.436 INFO:teuthology.orchestra.run.smithi007.stdout: thermald 2023-10-06T11:09:06.497 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2023-10-06T11:09:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T11:09:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-06T11:09:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-06T11:09:06.499 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2023-10-06T11:09:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-06T11:09:06.526 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:09:06.550 INFO:teuthology.orchestra.run.smithi007.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T11:09:06.550 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 77.0 MB of archives. 2023-10-06T11:09:06.550 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-06T11:09:06.551 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-06T11:09:06.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T11:09:06.623 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:09:06.813 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-10-06T11:09:06.813 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-10-06T11:09:06.869 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-06T11:09:06.959 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-06T11:09:06.996 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2023-10-06T11:09:06.996 INFO:teuthology.orchestra.run.smithi050.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T11:09:06.996 INFO:teuthology.orchestra.run.smithi050.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-06T11:09:06.996 INFO:teuthology.orchestra.run.smithi050.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-06T11:09:06.997 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2023-10-06T11:09:06.997 INFO:teuthology.orchestra.run.smithi050.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-06T11:09:06.998 INFO:teuthology.orchestra.run.smithi050.stdout:Recommended packages: 2023-10-06T11:09:06.998 INFO:teuthology.orchestra.run.smithi050.stdout: thermald 2023-10-06T11:09:07.051 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2023-10-06T11:09:07.051 INFO:teuthology.orchestra.run.smithi050.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T11:09:07.051 INFO:teuthology.orchestra.run.smithi050.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-06T11:09:07.051 INFO:teuthology.orchestra.run.smithi050.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-06T11:09:07.052 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2023-10-06T11:09:07.053 INFO:teuthology.orchestra.run.smithi050.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-06T11:09:07.235 INFO:teuthology.orchestra.run.smithi050.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T11:09:07.236 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 77.0 MB of archives. 2023-10-06T11:09:07.236 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-06T11:09:07.236 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-06T11:09:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-06T11:09:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-06T11:09:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-06T11:09:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-06T11:09:07.538 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-06T11:09:08.236 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 77.0 MB in 1s (74.6 MB/s) 2023-10-06T11:09:08.405 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-06T11:09:08.418 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-06T11:09:08.676 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-06T11:09:09.976 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-06T11:09:09.977 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-06T11:09:09.977 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-06T11:09:10.184 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-06T11:09:10.190 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:10.268 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:10.291 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-06T11:09:10.330 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-06T11:09:11.091 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 77.0 MB in 3s (23.6 MB/s) 2023-10-06T11:09:11.297 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-06T11:09:12.152 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-06T11:09:12.166 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:12.273 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:13.048 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-06T11:09:13.053 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:13.203 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:13.339 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-06T11:09:13.354 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:13.397 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:15.171 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-06T11:09:15.188 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:15.303 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-06T11:09:16.422 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:16.483 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:18.046 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:18.171 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:18.540 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:18.667 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:19.003 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-06T11:09:19.018 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-06T11:09:19.061 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T11:09:21.522 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:21.649 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:22.018 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:22.144 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:22.481 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-06T11:09:22.497 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-06T11:09:22.574 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T11:09:24.357 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-06T11:09:24.378 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:27.196 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:27.301 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:27.638 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T11:09:27.789 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:27.915 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:28.163 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-06T11:09:28.186 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T11:09:28.230 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:29.167 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:30.354 INFO:teuthology.orchestra.run.smithi007.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:09:30.354 INFO:teuthology.orchestra.run.smithi007.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-06T11:09:30.502 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-06T11:09:30.662 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:31.863 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-06T11:09:32.014 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-06T11:09:32.149 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:32.232 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-06T11:09:32.232 INFO:teuthology.orchestra.run.smithi007.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:09:32.358 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T11:09:32.527 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T11:09:32.872 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T11:09:33.000 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:33.126 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:34.387 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:35.572 INFO:teuthology.orchestra.run.smithi050.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:09:35.572 INFO:teuthology.orchestra.run.smithi050.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-06T11:09:35.689 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-06T11:09:35.817 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:38.092 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-06T11:09:38.218 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-06T11:09:38.370 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T11:09:38.453 INFO:teuthology.orchestra.run.smithi050.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-06T11:09:38.453 INFO:teuthology.orchestra.run.smithi050.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:09:55.046 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-06T11:09:55.047 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub' 2023-10-06T11:09:55.057 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:09:55.081 INFO:teuthology.orchestra.run.smithi007.stdout:Generating grub configuration file ... 2023-10-06T11:09:55.358 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:09:55.385 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:09:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:09:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:09:58.374 INFO:teuthology.orchestra.run.smithi007.stdout:done 2023-10-06T11:09:58.786 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2023-10-06T11:09:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2023-10-06T11:09:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2023-10-06T11:09:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2023-10-06T11:09:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2023-10-06T11:09:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 20 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.4.0.164.161 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T11:09:58.831 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T11:09:58.832 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2023-10-06T11:09:58.832 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2023-10-06T11:09:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2023-10-06T11:09:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2023-10-06T11:09:58.832 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-06T11:09:58.833 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T11:09:58.833 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T11:09:58.833 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T11:09:58.833 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T11:09:58.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2023-10-06T11:09:58.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2023-10-06T11:09:59.091 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:09:59.091 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:09:59.096 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:09:59.114 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:10:00.883 INFO:teuthology.orchestra.run.smithi050.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-06T11:10:00.883 INFO:teuthology.orchestra.run.smithi050.stdout:Sourcing file `/etc/default/grub' 2023-10-06T11:10:00.896 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:10:00.900 INFO:teuthology.orchestra.run.smithi050.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:10:00.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T11:10:00.965 INFO:teuthology.orchestra.run.smithi050.stdout:Generating grub configuration file ... 2023-10-06T11:10:00.995 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:10:01.224 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2023-10-06T11:10:01.225 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2023-10-06T11:10:01.347 INFO:teuthology.orchestra.run.smithi050.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:10:01.368 INFO:teuthology.orchestra.run.smithi050.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:10:01.461 INFO:teuthology.orchestra.run.smithi007.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-06T11:10:01.462 INFO:teuthology.orchestra.run.smithi007.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T11:10:01.463 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 20 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2023-10-06T11:10:01.484 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.4.0.164.161 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2023-10-06T11:10:01.485 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2023-10-06T11:10:01.485 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-10-06T11:10:01.490 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.o4tqcJQrJN 2023-10-06T11:10:01.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub/grub.cfg /tmp/tmp.o4tqcJQrJN 2023-10-06T11:10:01.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.o4tqcJQrJN 2023-10-06T11:10:01.603 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.o4tqcJQrJN is 9KB 2023-10-06T11:10:01.617 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.o4tqcJQrJN 2023-10-06T11:10:01.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-06T11:10:01.680 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:10:01.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-06T11:10:01.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-06T11:10:01.712 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-06T11:10:01.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo update-grub 2023-10-06T11:10:01.856 INFO:teuthology.orchestra.run.smithi050.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:10:01.867 INFO:teuthology.orchestra.run.smithi050.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:10:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:10:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:10:03.013 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2023-10-06T11:10:03.267 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:10:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:10:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:10:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:10:04.578 INFO:teuthology.orchestra.run.smithi050.stdout:done 2023-10-06T11:10:05.040 DEBUG:teuthology.orchestra.run.smithi050:> dpkg -s linux-image-generic 2023-10-06T11:10:05.090 INFO:teuthology.orchestra.run.smithi050.stdout:Package: linux-image-generic 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Status: install ok installed 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Priority: optional 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Section: kernel 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Installed-Size: 20 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: amd64 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Source: linux-meta 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Version: 5.4.0.164.161 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T11:10:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T11:10:05.092 INFO:teuthology.orchestra.run.smithi050.stdout:Recommends: thermald 2023-10-06T11:10:05.092 INFO:teuthology.orchestra.run.smithi050.stdout:Description: Generic Linux kernel image 2023-10-06T11:10:05.092 INFO:teuthology.orchestra.run.smithi050.stdout: This package will always depend on the latest generic kernel image 2023-10-06T11:10:05.092 INFO:teuthology.orchestra.run.smithi050.stdout: available. 2023-10-06T11:10:05.092 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-06T11:10:05.092 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T11:10:05.092 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T11:10:05.092 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T11:10:05.092 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T11:10:05.093 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get clean 2023-10-06T11:10:05.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-10-06T11:10:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:done 2023-10-06T11:10:05.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2023-10-06T11:10:05.302 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:10:05.308 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:10:05.325 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:10:05.440 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:10:07.091 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:10:07.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T11:10:07.186 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:10:07.394 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-10-06T11:10:07.394 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-10-06T11:10:07.616 INFO:teuthology.orchestra.run.smithi050.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-06T11:10:07.616 INFO:teuthology.orchestra.run.smithi050.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T11:10:07.617 DEBUG:teuthology.orchestra.run.smithi050:> dpkg -s linux-image-generic 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Package: linux-image-generic 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Status: install ok installed 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Priority: optional 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Section: kernel 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Installed-Size: 20 2023-10-06T11:10:07.635 INFO:teuthology.orchestra.run.smithi050.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: amd64 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Source: linux-meta 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Version: 5.4.0.164.161 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Recommends: thermald 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout:Description: Generic Linux kernel image 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: This package will always depend on the latest generic kernel image 2023-10-06T11:10:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: available. 2023-10-06T11:10:07.637 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-10-06T11:10:07.643 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.UHZUOgwBCh 2023-10-06T11:10:07.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub/grub.cfg /tmp/tmp.UHZUOgwBCh 2023-10-06T11:10:07.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.UHZUOgwBCh 2023-10-06T11:10:07.749 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.UHZUOgwBCh is 9KB 2023-10-06T11:10:07.763 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.UHZUOgwBCh 2023-10-06T11:10:07.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-06T11:10:07.828 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:10:07.828 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-06T11:10:07.828 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-06T11:10:07.855 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-06T11:10:07.855 DEBUG:teuthology.orchestra.run.smithi050:> sudo update-grub 2023-10-06T11:10:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:10:09.216 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:10:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2023-10-06T11:10:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T11:10:09.582 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T11:10:09.976 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:10:09.988 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:10:11.652 INFO:teuthology.orchestra.run.smithi050.stderr:done 2023-10-06T11:10:11.653 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2023-10-06T11:10:35.306 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T11:10:35.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:10:35.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:10:41.662 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T11:10:41.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:10:41.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:10:53.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-10-06T11:11:00.221 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-10-06T11:11:02.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:11:02.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:11:09.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:11:09.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:12:02.878 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T11:12:09.234 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T11:12:14.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:12:14.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:12:15.087 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-10-06T11:12:15.708 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:12:15.708 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-164-generic"... 2023-10-06T11:12:15.708 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-10-06T11:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:5.4.0-164-generic 2023-10-06T11:12:15.757 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-06T11:12:15.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:12:15.757 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T11:12:16.758 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T11:12:16.759 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T11:12:16.845 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2023-10-06T11:12:16.878 DEBUG:teuthology.parallel:result is None 2023-10-06T11:12:21.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:12:21.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:12:21.431 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-10-06T11:12:22.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:12:22.015 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-164-generic"... 2023-10-06T11:12:22.015 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-10-06T11:12:22.022 INFO:teuthology.orchestra.run.smithi050.stdout:5.4.0-164-generic 2023-10-06T11:12:22.022 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-06T11:12:22.022 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:12:22.022 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T11:12:23.023 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T11:12:23.024 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T11:12:23.113 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2023-10-06T11:12:23.148 DEBUG:teuthology.parallel:result is None 2023-10-06T11:12:23.149 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T11:12:23.161 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T11:12:23.161 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:12:23.163 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:12:23.170 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T11:12:23.182 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T11:12:23.201 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T11:12:23.201 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:12:23.204 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:12:23.242 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T11:12:23.254 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T11:12:23.254 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:12:23.256 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:12:23.272 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:12:23.282 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:12:23.295 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:12:23.306 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:12:23.308 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T11:12:23.319 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T11:12:23.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:12:23.333 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:12:23.350 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T11:12:23.363 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T11:12:23.363 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:12:23.365 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:12:23.372 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:12:23.379 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:12:23.432 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:12:23.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:12:23.454 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:12:23.461 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:12:23.468 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:12:23.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:12:23.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2023-10-06T11:12:23.497 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2023-10-06T11:12:23.547 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T11:12:23.558 INFO:teuthology.task.internal:Starting timer... 2023-10-06T11:12:23.559 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T11:12:23.579 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T11:12:23.589 DEBUG:teuthology.task.selinux:Excluding smithi007: OS 'ubuntu' does not support SELinux 2023-10-06T11:12:23.589 DEBUG:teuthology.task.selinux:Excluding smithi050: OS 'ubuntu' does not support SELinux 2023-10-06T11:12:23.590 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T11:12:23.590 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-06T11:12:23.590 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T11:12:23.590 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T11:12:23.603 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T11:12:23.608 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T11:12:23.735 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T11:12:23.750 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T11:12:23.751 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T11:18:07.146 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com')] 2023-10-06T11:18:07.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:18:07.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:18:07.197 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-10-06T11:18:07.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-06T11:18:07.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:18:07.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:18:07.373 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-10-06T11:18:07.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-06T11:18:07.496 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T11:18:07.508 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T11:18:07.508 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:18:07.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:18:07.511 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:18:07.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:18:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T11:18:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Command line: ntpd -gq 2023-10-06T11:18:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: proto: precision = 0.065 usec (-24) 2023-10-06T11:18:07.545 INFO:teuthology.orchestra.run.smithi007.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:18:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:18:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:18:07.546 INFO:teuthology.orchestra.run.smithi007.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen normally on 3 ens1f0 172.21.15.7:123 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen normally on 4 lo [::1]:123 2023-10-06T11:18:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15ea%3]:123 2023-10-06T11:18:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:07 ntpd[22765]: Listening on routing socket on fd #22 for interface updates 2023-10-06T11:18:07.548 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T11:18:07.548 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Command line: ntpd -gq 2023-10-06T11:18:07.548 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: proto: precision = 0.144 usec (-23) 2023-10-06T11:18:07.549 INFO:teuthology.orchestra.run.smithi050.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:18:07.549 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:18:07.549 INFO:teuthology.orchestra.run.smithi050.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:18:07.550 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:18:07.550 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T11:18:07.550 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T11:18:07.550 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T11:18:07.551 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen normally on 3 ens1f0 172.21.15.50:123 2023-10-06T11:18:07.551 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen normally on 4 lo [::1]:123 2023-10-06T11:18:07.551 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1584%4]:123 2023-10-06T11:18:07.551 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:07 ntpd[22857]: Listening on routing socket on fd #22 for interface updates 2023-10-06T11:18:08.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:08.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:08 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:08.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:08 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:09.545 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:09 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:09.545 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:09 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:09.546 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:09 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:09.546 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:09 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:10.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:10 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:10.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:10 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.549 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.549 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.549 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.549 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:12 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:12.552 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:12 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:14.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.546 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stdout: 6 Oct 11:18:14 ntpd[22765]: ntpd: time slew +0.000213 s 2023-10-06T11:18:14.547 INFO:teuthology.orchestra.run.smithi007.stdout:ntpd: time slew +0.000213s 2023-10-06T11:18:14.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T11:18:14.548 INFO:teuthology.orchestra.run.smithi007.stderr: 6 Oct 11:18:14 ntpd[22765]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T11:18:14.549 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.549 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.549 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:18:14.550 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:18:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: 6 Oct 11:18:14 ntpd[22857]: ntpd: time slew -0.001800 s 2023-10-06T11:18:14.551 INFO:teuthology.orchestra.run.smithi050.stdout:ntpd: time slew -0.001800s 2023-10-06T11:18:14.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T11:18:14.551 INFO:teuthology.orchestra.run.smithi050.stderr: 6 Oct 11:18:14 ntpd[22857]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.617 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:18:14.618 INFO:teuthology.run_tasks:Running task install... 2023-10-06T11:18:14.632 DEBUG:teuthology.task.install:project ceph 2023-10-06T11:18:14.632 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T11:18:14.632 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T11:18:14.633 INFO:teuthology.task.install:Using flavor: default 2023-10-06T11:18:14.639 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T11:18:14.639 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T11:18:14.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-key list | grep Ceph 2023-10-06T11:18:14.640 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-key list | grep Ceph 2023-10-06T11:18:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T11:18:14.747 INFO:teuthology.orchestra.run.smithi050.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T11:18:14.806 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T11:18:14.806 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T11:18:14.807 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T11:18:14.807 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T11:18:14.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:18:14.813 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T11:18:14.813 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T11:18:14.814 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T11:18:14.814 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T11:18:14.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:18:15.020 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default/ 2023-10-06T11:18:15.021 INFO:teuthology.task.install.deb:Package version is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:18:15.050 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default/ 2023-10-06T11:18:15.051 INFO:teuthology.task.install.deb:Package version is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:18:15.186 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:18:15.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T11:18:15.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2023-10-06T11:18:15.224 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:18:15.225 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T11:18:15.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-10-06T11:18:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:18:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:18:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:18:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:18:15.379 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:18:15.379 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:18:15.386 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:18:15.401 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:18:15.460 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal InRelease 2023-10-06T11:18:15.497 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal InRelease 2023-10-06T11:18:15.497 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T11:18:15.533 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T11:18:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T11:18:15.572 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T11:18:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main i386 Packages [3,391 B] 2023-10-06T11:18:15.608 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main i386 Packages [3,391 B] 2023-10-06T11:18:15.648 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T11:18:15.671 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T11:18:16.019 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 28.4 kB in 1s (44.0 kB/s) 2023-10-06T11:18:16.048 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 28.4 kB in 1s (44.3 kB/s) 2023-10-06T11:18:17.084 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:18:17.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1274-g68e53a4a-1focal cephadm=17.2.6-1274-g68e53a4a-1focal ceph-mds=17.2.6-1274-g68e53a4a-1focal ceph-mgr=17.2.6-1274-g68e53a4a-1focal ceph-common=17.2.6-1274-g68e53a4a-1focal ceph-fuse=17.2.6-1274-g68e53a4a-1focal ceph-test=17.2.6-1274-g68e53a4a-1focal ceph-volume=17.2.6-1274-g68e53a4a-1focal radosgw=17.2.6-1274-g68e53a4a-1focal python3-rados=17.2.6-1274-g68e53a4a-1focal python3-rgw=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal python3-rbd=17.2.6-1274-g68e53a4a-1focal libcephfs2=17.2.6-1274-g68e53a4a-1focal libcephfs-dev=17.2.6-1274-g68e53a4a-1focal librados2=17.2.6-1274-g68e53a4a-1focal librbd1=17.2.6-1274-g68e53a4a-1focal rbd-fuse=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal cephfs-shell=17.2.6-1274-g68e53a4a-1focal cephfs-top=17.2.6-1274-g68e53a4a-1focal cephfs-mirror=17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:18:17.124 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:18:17.151 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1274-g68e53a4a-1focal cephadm=17.2.6-1274-g68e53a4a-1focal ceph-mds=17.2.6-1274-g68e53a4a-1focal ceph-mgr=17.2.6-1274-g68e53a4a-1focal ceph-common=17.2.6-1274-g68e53a4a-1focal ceph-fuse=17.2.6-1274-g68e53a4a-1focal ceph-test=17.2.6-1274-g68e53a4a-1focal ceph-volume=17.2.6-1274-g68e53a4a-1focal radosgw=17.2.6-1274-g68e53a4a-1focal python3-rados=17.2.6-1274-g68e53a4a-1focal python3-rgw=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal python3-rbd=17.2.6-1274-g68e53a4a-1focal libcephfs2=17.2.6-1274-g68e53a4a-1focal libcephfs-dev=17.2.6-1274-g68e53a4a-1focal librados2=17.2.6-1274-g68e53a4a-1focal librbd1=17.2.6-1274-g68e53a4a-1focal rbd-fuse=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal cephfs-shell=17.2.6-1274-g68e53a4a-1focal cephfs-top=17.2.6-1274-g68e53a4a-1focal cephfs-mirror=17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:18:17.177 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:18:17.224 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:18:17.361 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2023-10-06T11:18:17.362 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2023-10-06T11:18:17.409 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-10-06T11:18:17.410 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-10-06T11:18:17.540 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T11:18:17.542 INFO:teuthology.orchestra.run.smithi007.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T11:18:17.543 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T11:18:17.543 INFO:teuthology.orchestra.run.smithi007.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T11:18:17.543 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:18:17.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T11:18:17.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2023-10-06T11:18:17.546 INFO:teuthology.orchestra.run.smithi007.stdout: btrfs-tools 2023-10-06T11:18:17.591 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:18:17.592 INFO:teuthology.orchestra.run.smithi050.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:18:17.592 INFO:teuthology.orchestra.run.smithi050.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T11:18:17.593 INFO:teuthology.orchestra.run.smithi050.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:18:17.594 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:18:17.595 INFO:teuthology.orchestra.run.smithi050.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T11:18:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T11:18:17.597 INFO:teuthology.orchestra.run.smithi050.stdout:Recommended packages: 2023-10-06T11:18:17.597 INFO:teuthology.orchestra.run.smithi050.stdout: btrfs-tools 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T11:18:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T11:18:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T11:18:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T11:18:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T11:18:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2023-10-06T11:18:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 librbd1 2023-10-06T11:18:17.724 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2023-10-06T11:18:17.724 INFO:teuthology.orchestra.run.smithi050.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T11:18:17.724 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T11:18:17.724 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T11:18:17.725 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T11:18:17.725 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T11:18:17.725 INFO:teuthology.orchestra.run.smithi050.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T11:18:17.725 INFO:teuthology.orchestra.run.smithi050.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T11:18:17.726 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T11:18:17.726 INFO:teuthology.orchestra.run.smithi050.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T11:18:17.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:18:17.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T11:18:17.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:18:17.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:18:17.728 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:18:17.728 INFO:teuthology.orchestra.run.smithi050.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T11:18:17.728 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2023-10-06T11:18:17.728 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 librbd1 2023-10-06T11:18:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-06T11:18:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 210 MB of archives. 2023-10-06T11:18:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T11:18:17.742 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T11:18:17.763 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T11:18:17.772 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T11:18:17.795 INFO:teuthology.orchestra.run.smithi050.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-06T11:18:17.795 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 210 MB of archives. 2023-10-06T11:18:17.795 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T11:18:17.796 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T11:18:17.802 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T11:18:17.814 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T11:18:17.819 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1274-g68e53a4a-1focal [3,501 kB] 2023-10-06T11:18:17.821 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T11:18:17.836 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T11:18:17.839 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T11:18:17.847 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T11:18:17.851 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T11:18:17.853 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T11:18:17.874 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T11:18:17.875 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T11:18:17.876 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T11:18:17.878 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T11:18:17.881 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T11:18:17.887 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T11:18:17.890 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T11:18:17.908 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T11:18:17.913 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T11:18:17.920 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T11:18:17.922 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T11:18:17.931 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T11:18:17.936 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T11:18:17.936 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T11:18:17.938 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T11:18:17.939 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T11:18:17.939 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T11:18:17.940 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T11:18:17.941 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T11:18:17.942 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T11:18:17.948 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T11:18:17.950 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T11:18:17.950 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T11:18:17.952 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T11:18:17.953 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T11:18:17.953 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T11:18:17.954 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T11:18:17.954 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T11:18:17.955 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T11:18:17.962 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T11:18:17.963 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T11:18:17.970 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1274-g68e53a4a-1focal [3,501 kB] 2023-10-06T11:18:17.974 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T11:18:17.974 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T11:18:17.975 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T11:18:17.976 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T11:18:17.976 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T11:18:17.978 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T11:18:17.979 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T11:18:17.979 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T11:18:17.980 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T11:18:17.981 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T11:18:17.981 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T11:18:17.982 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T11:18:17.982 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T11:18:17.984 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T11:18:17.984 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T11:18:17.986 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T11:18:17.993 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T11:18:17.994 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T11:18:17.996 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T11:18:17.996 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T11:18:18.013 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T11:18:18.014 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T11:18:18.014 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T11:18:18.014 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T11:18:18.015 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T11:18:18.015 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T11:18:18.015 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T11:18:18.015 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T11:18:18.016 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T11:18:18.017 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T11:18:18.019 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T11:18:18.021 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T11:18:18.021 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T11:18:18.021 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T11:18:18.022 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T11:18:18.023 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T11:18:18.030 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T11:18:18.031 INFO:teuthology.orchestra.run.smithi050.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T11:18:18.033 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T11:18:18.034 INFO:teuthology.orchestra.run.smithi007.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T11:18:18.047 INFO:teuthology.orchestra.run.smithi050.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T11:18:18.048 INFO:teuthology.orchestra.run.smithi050.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T11:18:18.048 INFO:teuthology.orchestra.run.smithi050.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T11:18:18.053 INFO:teuthology.orchestra.run.smithi007.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T11:18:18.054 INFO:teuthology.orchestra.run.smithi007.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T11:18:18.055 INFO:teuthology.orchestra.run.smithi007.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T11:18:18.090 INFO:teuthology.orchestra.run.smithi007.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1274-g68e53a4a-1focal [3,551 kB] 2023-10-06T11:18:18.133 INFO:teuthology.orchestra.run.smithi050.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T11:18:18.149 INFO:teuthology.orchestra.run.smithi050.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T11:18:18.150 INFO:teuthology.orchestra.run.smithi050.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T11:18:18.162 INFO:teuthology.orchestra.run.smithi050.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T11:18:18.162 INFO:teuthology.orchestra.run.smithi050.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T11:18:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1274-g68e53a4a-1focal [760 kB] 2023-10-06T11:18:18.225 INFO:teuthology.orchestra.run.smithi050.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T11:18:18.226 INFO:teuthology.orchestra.run.smithi050.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T11:18:18.226 INFO:teuthology.orchestra.run.smithi050.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T11:18:18.227 INFO:teuthology.orchestra.run.smithi050.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T11:18:18.228 INFO:teuthology.orchestra.run.smithi050.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T11:18:18.229 INFO:teuthology.orchestra.run.smithi050.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T11:18:18.229 INFO:teuthology.orchestra.run.smithi050.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T11:18:18.242 INFO:teuthology.orchestra.run.smithi050.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T11:18:18.246 INFO:teuthology.orchestra.run.smithi050.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T11:18:18.249 INFO:teuthology.orchestra.run.smithi050.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T11:18:18.259 INFO:teuthology.orchestra.run.smithi050.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T11:18:18.260 INFO:teuthology.orchestra.run.smithi050.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T11:18:18.262 INFO:teuthology.orchestra.run.smithi050.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T11:18:18.265 INFO:teuthology.orchestra.run.smithi050.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T11:18:18.266 INFO:teuthology.orchestra.run.smithi050.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T11:18:18.283 INFO:teuthology.orchestra.run.smithi050.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi050.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi007.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1274-g68e53a4a-1focal [413 kB] 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi007.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T11:18:18.305 INFO:teuthology.orchestra.run.smithi007.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1274-g68e53a4a-1focal [22.5 kB] 2023-10-06T11:18:18.306 INFO:teuthology.orchestra.run.smithi007.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1274-g68e53a4a-1focal [200 kB] 2023-10-06T11:18:18.308 INFO:teuthology.orchestra.run.smithi007.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1274-g68e53a4a-1focal [47.1 kB] 2023-10-06T11:18:18.308 INFO:teuthology.orchestra.run.smithi007.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1274-g68e53a4a-1focal [386 kB] 2023-10-06T11:18:18.312 INFO:teuthology.orchestra.run.smithi007.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1274-g68e53a4a-1focal [5,567 kB] 2023-10-06T11:18:18.320 INFO:teuthology.orchestra.run.smithi007.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T11:18:18.322 INFO:teuthology.orchestra.run.smithi007.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T11:18:18.322 INFO:teuthology.orchestra.run.smithi050.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T11:18:18.323 INFO:teuthology.orchestra.run.smithi050.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T11:18:18.323 INFO:teuthology.orchestra.run.smithi050.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T11:18:18.323 INFO:teuthology.orchestra.run.smithi050.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T11:18:18.324 INFO:teuthology.orchestra.run.smithi050.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T11:18:18.325 INFO:teuthology.orchestra.run.smithi050.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T11:18:18.336 INFO:teuthology.orchestra.run.smithi050.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T11:18:18.337 INFO:teuthology.orchestra.run.smithi050.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T11:18:18.337 INFO:teuthology.orchestra.run.smithi050.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T11:18:18.339 INFO:teuthology.orchestra.run.smithi050.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T11:18:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T11:18:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T11:18:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T11:18:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T11:18:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T11:18:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T11:18:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T11:18:18.356 INFO:teuthology.orchestra.run.smithi050.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T11:18:18.357 INFO:teuthology.orchestra.run.smithi050.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T11:18:18.358 INFO:teuthology.orchestra.run.smithi050.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T11:18:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T11:18:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T11:18:18.360 INFO:teuthology.orchestra.run.smithi050.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T11:18:18.360 INFO:teuthology.orchestra.run.smithi050.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T11:18:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T11:18:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T11:18:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T11:18:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T11:18:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T11:18:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T11:18:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T11:18:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T11:18:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T11:18:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T11:18:18.528 INFO:teuthology.orchestra.run.smithi007.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1274-g68e53a4a-1focal [116 kB] 2023-10-06T11:18:18.534 INFO:teuthology.orchestra.run.smithi007.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1274-g68e53a4a-1focal [479 kB] 2023-10-06T11:18:18.535 INFO:teuthology.orchestra.run.smithi007.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T11:18:18.607 INFO:teuthology.orchestra.run.smithi007.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T11:18:18.607 INFO:teuthology.orchestra.run.smithi007.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T11:18:18.607 INFO:teuthology.orchestra.run.smithi007.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1274-g68e53a4a-1focal [27.7 MB] 2023-10-06T11:18:18.608 INFO:teuthology.orchestra.run.smithi007.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T11:18:18.608 INFO:teuthology.orchestra.run.smithi007.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T11:18:18.609 INFO:teuthology.orchestra.run.smithi007.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T11:18:18.628 INFO:teuthology.orchestra.run.smithi007.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T11:18:18.628 INFO:teuthology.orchestra.run.smithi007.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T11:18:18.629 INFO:teuthology.orchestra.run.smithi007.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T11:18:18.629 INFO:teuthology.orchestra.run.smithi007.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T11:18:18.634 INFO:teuthology.orchestra.run.smithi007.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T11:18:18.635 INFO:teuthology.orchestra.run.smithi007.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T11:18:18.635 INFO:teuthology.orchestra.run.smithi007.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T11:18:18.660 INFO:teuthology.orchestra.run.smithi007.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T11:18:18.661 INFO:teuthology.orchestra.run.smithi007.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T11:18:18.758 INFO:teuthology.orchestra.run.smithi050.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1274-g68e53a4a-1focal [3,551 kB] 2023-10-06T11:18:19.270 INFO:teuthology.orchestra.run.smithi050.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1274-g68e53a4a-1focal [760 kB] 2023-10-06T11:18:19.396 INFO:teuthology.orchestra.run.smithi050.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1274-g68e53a4a-1focal [413 kB] 2023-10-06T11:18:19.518 INFO:teuthology.orchestra.run.smithi050.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1274-g68e53a4a-1focal [22.5 kB] 2023-10-06T11:18:19.519 INFO:teuthology.orchestra.run.smithi050.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1274-g68e53a4a-1focal [200 kB] 2023-10-06T11:18:19.524 INFO:teuthology.orchestra.run.smithi050.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1274-g68e53a4a-1focal [47.1 kB] 2023-10-06T11:18:19.525 INFO:teuthology.orchestra.run.smithi050.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1274-g68e53a4a-1focal [386 kB] 2023-10-06T11:18:19.554 INFO:teuthology.orchestra.run.smithi050.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1274-g68e53a4a-1focal [5,567 kB] 2023-10-06T11:18:19.776 INFO:teuthology.orchestra.run.smithi007.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1274-g68e53a4a-1focal [6,449 kB] 2023-10-06T11:18:19.969 INFO:teuthology.orchestra.run.smithi007.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1274-g68e53a4a-1focal [221 kB] 2023-10-06T11:18:19.978 INFO:teuthology.orchestra.run.smithi007.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1274-g68e53a4a-1focal [152 kB] 2023-10-06T11:18:19.984 INFO:teuthology.orchestra.run.smithi007.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1274-g68e53a4a-1focal [1,590 kB] 2023-10-06T11:18:20.053 INFO:teuthology.orchestra.run.smithi007.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1274-g68e53a4a-1focal [7,846 kB] 2023-10-06T11:18:20.371 INFO:teuthology.orchestra.run.smithi050.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1274-g68e53a4a-1focal [116 kB] 2023-10-06T11:18:20.382 INFO:teuthology.orchestra.run.smithi050.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1274-g68e53a4a-1focal [479 kB] 2023-10-06T11:18:20.388 INFO:teuthology.orchestra.run.smithi007.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1274-g68e53a4a-1focal [28.7 MB] 2023-10-06T11:18:20.458 INFO:teuthology.orchestra.run.smithi050.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1274-g68e53a4a-1focal [27.7 MB] 2023-10-06T11:18:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1274-g68e53a4a-1focal [3,928 B] 2023-10-06T11:18:21.782 INFO:teuthology.orchestra.run.smithi007.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1274-g68e53a4a-1focal [870 kB] 2023-10-06T11:18:21.796 INFO:teuthology.orchestra.run.smithi007.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1274-g68e53a4a-1focal [2,406 kB] 2023-10-06T11:18:21.814 INFO:teuthology.orchestra.run.smithi007.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1274-g68e53a4a-1focal [79.1 kB] 2023-10-06T11:18:21.814 INFO:teuthology.orchestra.run.smithi007.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1274-g68e53a4a-1focal [117 kB] 2023-10-06T11:18:21.815 INFO:teuthology.orchestra.run.smithi007.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1274-g68e53a4a-1focal [1,760 kB] 2023-10-06T11:18:21.855 INFO:teuthology.orchestra.run.smithi007.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1274-g68e53a4a-1focal [7,886 kB] 2023-10-06T11:18:22.186 INFO:teuthology.orchestra.run.smithi007.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1274-g68e53a4a-1focal [4,040 B] 2023-10-06T11:18:22.186 INFO:teuthology.orchestra.run.smithi007.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1274-g68e53a4a-1focal [56.3 MB] 2023-10-06T11:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1274-g68e53a4a-1focal [6,449 kB] 2023-10-06T11:18:24.786 INFO:teuthology.orchestra.run.smithi007.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1274-g68e53a4a-1focal [229 kB] 2023-10-06T11:18:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1274-g68e53a4a-1focal [239 kB] 2023-10-06T11:18:24.796 INFO:teuthology.orchestra.run.smithi007.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1274-g68e53a4a-1focal [22.0 kB] 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi007.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1274-g68e53a4a-1focal [12.8 kB] 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi007.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1274-g68e53a4a-1focal [21.9 kB] 2023-10-06T11:18:24.855 INFO:teuthology.orchestra.run.smithi007.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1274-g68e53a4a-1focal [17.0 MB] 2023-10-06T11:18:25.261 INFO:teuthology.orchestra.run.smithi050.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1274-g68e53a4a-1focal [221 kB] 2023-10-06T11:18:25.296 INFO:teuthology.orchestra.run.smithi050.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1274-g68e53a4a-1focal [152 kB] 2023-10-06T11:18:25.317 INFO:teuthology.orchestra.run.smithi050.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1274-g68e53a4a-1focal [1,590 kB] 2023-10-06T11:18:25.471 INFO:teuthology.orchestra.run.smithi050.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1274-g68e53a4a-1focal [7,846 kB] 2023-10-06T11:18:25.482 INFO:teuthology.orchestra.run.smithi007.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1274-g68e53a4a-1focal [79.4 kB] 2023-10-06T11:18:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 210 MB in 8s (26.9 MB/s) 2023-10-06T11:18:26.272 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi050.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1274-g68e53a4a-1focal [28.7 MB] 2023-10-06T11:18:26.328 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-06T11:18:26.333 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T11:18:26.370 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T11:18:26.698 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T11:18:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T11:18:26.757 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:18:27.068 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T11:18:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T11:18:27.126 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:18:27.479 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T11:18:27.495 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T11:18:27.579 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:18:27.950 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../004-librbd1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librbd1 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:18:28.779 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../005-librados2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:28.905 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librados2 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:18:29.369 INFO:teuthology.orchestra.run.smithi050.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1274-g68e53a4a-1focal [3,928 B] 2023-10-06T11:18:29.370 INFO:teuthology.orchestra.run.smithi050.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1274-g68e53a4a-1focal [870 kB] 2023-10-06T11:18:29.385 INFO:teuthology.orchestra.run.smithi050.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1274-g68e53a4a-1focal [2,406 kB] 2023-10-06T11:18:29.551 INFO:teuthology.orchestra.run.smithi050.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1274-g68e53a4a-1focal [79.1 kB] 2023-10-06T11:18:29.555 INFO:teuthology.orchestra.run.smithi050.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1274-g68e53a4a-1focal [117 kB] 2023-10-06T11:18:29.559 INFO:teuthology.orchestra.run.smithi050.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1274-g68e53a4a-1focal [1,760 kB] 2023-10-06T11:18:29.592 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T11:18:29.612 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:29.667 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:29.702 INFO:teuthology.orchestra.run.smithi050.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1274-g68e53a4a-1focal [7,886 kB] 2023-10-06T11:18:29.987 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rados. 2023-10-06T11:18:30.005 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:30.036 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:30.389 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T11:18:30.406 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:30.448 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:30.479 INFO:teuthology.orchestra.run.smithi050.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1274-g68e53a4a-1focal [4,040 B] 2023-10-06T11:18:30.479 INFO:teuthology.orchestra.run.smithi050.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1274-g68e53a4a-1focal [56.3 MB] 2023-10-06T11:18:30.700 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T11:18:30.717 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:30.759 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:31.103 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T11:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:31.161 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:31.481 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T11:18:31.498 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T11:18:31.548 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T11:18:31.789 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T11:18:31.806 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:31.849 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:32.231 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T11:18:32.251 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:18:32.639 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T11:18:32.660 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T11:18:32.698 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:18:33.060 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T11:18:33.078 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T11:18:33.118 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:18:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T11:18:33.505 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T11:18:33.546 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:18:33.875 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T11:18:33.891 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T11:18:33.933 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:18:34.429 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T11:18:34.446 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:34.990 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T11:18:35.009 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:35.057 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:35.427 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T11:18:35.431 INFO:teuthology.orchestra.run.smithi050.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1274-g68e53a4a-1focal [229 kB] 2023-10-06T11:18:35.436 INFO:teuthology.orchestra.run.smithi050.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1274-g68e53a4a-1focal [239 kB] 2023-10-06T11:18:35.445 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:35.485 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:35.486 INFO:teuthology.orchestra.run.smithi050.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1274-g68e53a4a-1focal [22.0 kB] 2023-10-06T11:18:35.487 INFO:teuthology.orchestra.run.smithi050.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1274-g68e53a4a-1focal [12.8 kB] 2023-10-06T11:18:35.487 INFO:teuthology.orchestra.run.smithi050.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1274-g68e53a4a-1focal [21.9 kB] 2023-10-06T11:18:35.488 INFO:teuthology.orchestra.run.smithi050.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1274-g68e53a4a-1focal [17.0 MB] 2023-10-06T11:18:36.131 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T11:18:36.147 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T11:18:36.189 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:18:36.542 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librgw2. 2023-10-06T11:18:36.561 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../022-librgw2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:36.600 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:36.695 INFO:teuthology.orchestra.run.smithi050.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1274-g68e53a4a-1focal [79.4 kB] 2023-10-06T11:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 210 MB in 19s (11.1 MB/s) 2023-10-06T11:18:37.428 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T11:18:37.444 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:37.486 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T11:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-06T11:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T11:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T11:18:37.832 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liboath0. 2023-10-06T11:18:37.847 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T11:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T11:18:37.939 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T11:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:18:38.258 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T11:18:38.275 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T11:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T11:18:38.308 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:18:38.620 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T11:18:38.638 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T11:18:38.720 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:18:38.788 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-common. 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:38.846 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:39.099 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../004-librbd1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:39.246 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librbd1 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:18:39.962 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../005-librados2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:40.129 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librados2 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:18:40.834 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T11:18:40.850 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:40.883 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:41.211 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rados. 2023-10-06T11:18:41.227 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:41.269 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:41.560 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-base. 2023-10-06T11:18:41.580 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:41.629 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:41.647 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T11:18:41.663 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:41.705 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T11:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:42.041 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:42.377 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T11:18:42.395 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:42.436 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:42.657 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T11:18:42.673 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T11:18:42.715 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:18:42.733 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T11:18:42.748 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T11:18:42.783 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T11:18:43.052 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T11:18:43.072 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:43.110 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:43.119 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T11:18:43.135 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T11:18:43.177 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T11:18:43.430 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T11:18:43.449 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:43.488 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:43.494 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T11:18:43.553 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:18:43.843 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T11:18:43.861 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T11:18:43.901 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:18:43.939 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T11:18:43.955 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T11:18:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:18:44.188 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T11:18:44.204 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T11:18:44.271 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:18:44.336 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T11:18:44.352 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T11:18:44.395 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:18:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T11:18:44.619 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T11:18:44.658 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T11:18:44.806 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T11:18:44.822 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T11:18:44.864 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:18:44.995 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T11:18:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:18:45.192 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T11:18:45.208 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T11:18:45.242 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:18:45.348 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T11:18:45.367 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T11:18:45.398 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T11:18:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T11:18:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T11:18:45.714 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T11:18:45.732 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:45.750 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:18:45.781 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:46.079 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-mako. 2023-10-06T11:18:46.097 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T11:18:46.137 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:18:46.282 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T11:18:46.301 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:46.365 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:46.457 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T11:18:46.476 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T11:18:46.515 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:18:46.668 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T11:18:46.686 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:18:46.718 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:18:46.785 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T11:18:46.801 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T11:18:46.843 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:18:47.096 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T11:18:47.112 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T11:18:47.154 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webob. 2023-10-06T11:18:47.155 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:18:47.173 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T11:18:47.212 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T11:18:47.507 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librgw2. 2023-10-06T11:18:47.524 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../022-librgw2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:47.540 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T11:18:47.557 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T11:18:47.565 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:47.599 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:18:47.935 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T11:18:47.951 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T11:18:47.993 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T11:18:48.355 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T11:18:48.368 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T11:18:48.386 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:48.416 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:18:48.417 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:48.741 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T11:18:48.760 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T11:18:48.762 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liboath0. 2023-10-06T11:18:48.781 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T11:18:48.800 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T11:18:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T11:18:49.111 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-paste. 2023-10-06T11:18:49.127 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T11:18:49.165 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T11:18:49.182 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:49.205 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:18:49.224 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:49.498 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T11:18:49.522 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T11:18:49.556 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:18:49.661 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-common. 2023-10-06T11:18:49.682 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:49.711 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T11:18:51.138 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T11:18:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:18:52.190 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T11:18:52.206 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:52.248 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:52.585 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T11:18:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:52.790 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-base. 2023-10-06T11:18:52.809 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:52.868 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:53.121 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T11:18:53.140 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:53.179 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:53.862 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T11:18:53.883 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T11:18:53.920 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:18:54.316 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T11:18:54.334 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T11:18:54.342 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T11:18:54.358 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:54.374 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T11:18:54.399 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:54.677 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T11:18:54.692 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:54.734 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:55.150 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T11:18:55.165 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T11:18:55.206 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:18:55.534 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T11:18:55.552 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T11:18:55.592 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:18:55.971 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T11:18:55.990 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T11:18:56.062 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T11:18:56.424 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T11:18:56.444 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T11:18:56.482 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:18:56.827 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T11:18:56.845 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T11:18:56.885 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T11:18:57.221 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T11:18:57.236 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph. 2023-10-06T11:18:57.241 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T11:18:57.252 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../055-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:57.279 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:18:57.294 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:57.582 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-mako. 2023-10-06T11:18:57.598 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T11:18:57.613 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T11:18:57.630 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:57.632 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:18:57.696 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:57.985 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T11:18:58.002 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T11:18:58.052 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T11:18:58.141 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:58.183 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:58.355 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T11:18:58.373 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T11:18:58.405 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:18:58.649 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webob. 2023-10-06T11:18:58.666 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T11:18:58.707 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T11:18:58.728 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephadm. 2023-10-06T11:18:58.744 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../058-cephadm_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:18:58.786 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:59.044 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T11:18:59.062 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T11:18:59.101 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:18:59.114 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T11:18:59.133 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T11:18:59.172 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T11:18:59.413 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T11:18:59.488 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T11:18:59.500 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T11:18:59.520 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:18:59.558 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:18:59.855 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T11:18:59.874 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T11:18:59.927 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:18:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T11:18:59.947 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T11:18:59.988 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T11:19:00.250 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T11:19:00.291 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:19:00.294 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T11:19:00.313 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T11:19:00.352 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T11:19:00.593 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T11:19:00.646 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-paste. 2023-10-06T11:19:00.651 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:19:00.663 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T11:19:00.721 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:19:00.971 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-routes. 2023-10-06T11:19:00.989 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T11:19:01.029 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T11:19:01.033 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T11:19:01.050 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T11:19:01.124 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:19:01.382 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T11:19:01.398 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:01.427 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T11:19:01.440 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:01.443 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T11:19:01.485 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:19:01.797 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T11:19:01.817 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T11:19:01.855 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T11:19:01.936 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T11:19:01.952 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T11:19:01.985 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:19:02.191 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T11:19:02.207 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T11:19:02.249 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:19:02.297 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T11:19:02.314 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T11:19:02.355 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:19:02.629 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T11:19:02.646 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T11:19:02.687 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:19:02.716 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T11:19:02.733 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T11:19:02.774 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T11:19:03.057 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T11:19:03.077 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:03.493 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T11:19:03.510 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:03.551 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:04.063 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T11:19:04.067 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T11:19:04.081 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:04.085 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T11:19:04.121 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:04.125 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:04.662 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T11:19:04.678 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T11:19:04.753 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T11:19:05.115 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T11:19:05.132 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T11:19:05.165 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:05.275 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T11:19:05.294 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:05.874 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T11:19:05.891 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:05.932 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:07.102 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T11:19:07.119 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T11:19:07.160 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T11:19:07.738 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T11:19:07.757 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T11:19:07.788 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:19:08.058 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T11:19:08.077 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T11:19:08.116 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:19:08.176 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph. 2023-10-06T11:19:08.192 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../055-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:08.244 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:08.402 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T11:19:08.421 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T11:19:08.461 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T11:19:08.564 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T11:19:08.582 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:08.638 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:08.773 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T11:19:08.790 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T11:19:08.831 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T11:19:09.041 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T11:19:09.058 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:09.099 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:09.176 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T11:19:09.192 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T11:19:09.234 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:19:09.487 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T11:19:09.505 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T11:19:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:19:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephadm. 2023-10-06T11:19:09.646 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../058-cephadm_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:09.686 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:09.907 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T11:19:09.923 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T11:19:09.965 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:19:10.039 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T11:19:10.055 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T11:19:10.097 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T11:19:10.354 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T11:19:10.371 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T11:19:10.442 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T11:19:10.445 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:19:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:10.499 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:10.757 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T11:19:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T11:19:10.807 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:19:10.869 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T11:19:10.889 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T11:19:10.936 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T11:19:11.222 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T11:19:11.239 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T11:19:11.280 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:19:11.355 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T11:19:11.374 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:11.608 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T11:19:11.626 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T11:19:11.666 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:19:11.707 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T11:19:11.724 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T11:19:11.766 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:19:12.010 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-routes. 2023-10-06T11:19:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T11:19:12.068 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T11:19:12.152 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T11:19:12.170 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:19:12.210 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:12.430 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:12.488 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:12.488 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package jq. 2023-10-06T11:19:12.508 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:19:12.571 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:12.791 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package socat. 2023-10-06T11:19:12.807 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T11:19:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T11:19:12.991 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T11:19:13.007 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T11:19:13.049 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:19:13.144 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T11:19:13.162 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T11:19:13.202 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:19:13.377 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T11:19:13.394 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T11:19:13.435 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:19:13.548 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-test. 2023-10-06T11:19:13.566 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:13.598 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:13.746 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T11:19:13.765 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T11:19:13.804 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T11:19:15.130 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T11:19:15.147 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T11:19:15.188 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:15.791 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T11:19:15.808 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T11:19:15.849 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T11:19:16.253 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T11:19:16.270 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T11:19:16.311 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:17.078 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T11:19:17.097 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:17.135 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:18.239 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T11:19:18.258 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T11:19:18.289 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T11:19:18.650 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T11:19:18.667 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T11:19:18.685 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T11:19:18.705 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:18.708 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:19:18.743 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:19.061 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T11:19:19.079 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T11:19:19.119 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:19:19.157 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T11:19:19.174 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:19.215 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:19.456 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T11:19:19.472 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T11:19:19.514 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T11:19:19.685 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T11:19:19.702 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T11:19:19.743 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T11:19:19.850 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T11:19:19.868 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T11:19:19.908 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T11:19:20.021 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T11:19:20.038 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:20.070 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:20.203 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T11:19:20.221 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T11:19:20.278 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:19:20.332 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T11:19:20.350 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:20.390 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:20.557 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T11:19:20.574 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T11:19:20.614 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:19:20.669 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T11:19:20.686 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:19:20.726 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:20.992 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T11:19:21.009 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T11:19:21.072 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:19:21.251 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T11:19:21.268 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:21.309 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:21.464 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T11:19:21.480 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T11:19:21.530 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:19:21.654 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T11:19:21.673 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T11:19:21.730 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:19:21.834 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T11:19:21.851 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T11:19:21.925 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:19:22.108 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T11:19:22.126 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:19:22.166 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:22.498 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T11:19:22.515 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:22.556 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:22.909 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T11:19:22.927 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T11:19:22.967 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:19:23.346 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T11:19:23.363 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:19:23.403 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package jq. 2023-10-06T11:19:23.724 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:19:23.764 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:23.891 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T11:19:23.909 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T11:19:23.949 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T11:19:24.059 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package socat. 2023-10-06T11:19:24.076 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T11:19:24.117 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T11:19:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T11:19:24.344 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:19:24.444 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T11:19:24.463 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T11:19:24.502 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:19:24.647 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-attr. 2023-10-06T11:19:24.666 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T11:19:24.729 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T11:19:24.874 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-test. 2023-10-06T11:19:24.891 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:24.932 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:25.043 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T11:19:25.062 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T11:19:25.101 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:19:25.379 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T11:19:25.454 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T11:19:25.798 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T11:19:25.815 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T11:19:25.856 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:19:26.303 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T11:19:26.321 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T11:19:26.360 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T11:19:26.622 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T11:19:26.639 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T11:19:26.671 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T11:19:26.966 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T11:19:26.985 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T11:19:27.016 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T11:19:27.328 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T11:19:27.346 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T11:19:27.377 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:19:27.739 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-py. 2023-10-06T11:19:27.757 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T11:19:27.797 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:19:28.134 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T11:19:28.151 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T11:19:28.201 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:19:28.537 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T11:19:28.557 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T11:19:28.595 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T11:19:28.969 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T11:19:28.987 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T11:19:29.027 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:19:29.304 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T11:19:29.323 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T11:19:29.362 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:19:30.003 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T11:19:30.020 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:30.052 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:30.092 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package radosgw. 2023-10-06T11:19:30.109 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../114-radosgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:30.149 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:30.507 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T11:19:30.524 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:30.943 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T11:19:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T11:19:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T11:19:31.296 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T11:19:31.314 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:31.354 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:31.641 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:19:31.698 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:31.980 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T11:19:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:19:32.038 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:32.169 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T11:19:32.187 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:32.227 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:32.567 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T11:19:32.567 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T11:19:32.585 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:32.625 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:32.875 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:19:33.001 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:19:33.035 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T11:19:33.054 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T11:19:33.111 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:19:33.127 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:19:33.357 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:33.440 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T11:19:33.458 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:19:33.498 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:33.969 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user cephadm....done 2023-10-06T11:19:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:34.264 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:19:34.627 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T11:19:34.961 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:35.087 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T11:19:35.160 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T11:19:35.178 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T11:19:35.205 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:19:35.210 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T11:19:35.488 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T11:19:35.507 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T11:19:35.537 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:19:35.548 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T11:19:35.849 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-attr. 2023-10-06T11:19:36.343 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:19:36.343 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T11:19:36.343 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T11:19:36.344 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T11:19:36.344 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T11:19:36.344 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T11:19:36.344 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:19:36.561 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T11:19:36.662 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T11:19:36.680 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T11:19:36.720 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T11:19:36.862 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:19:37.154 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:19:37.430 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:37.441 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T11:19:37.462 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T11:19:37.565 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:19:37.706 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:37.832 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T11:19:38.061 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T11:19:38.078 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T11:19:38.119 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T11:19:38.167 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:19:38.447 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T11:19:38.464 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T11:19:38.504 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T11:19:38.726 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:19:38.841 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T11:19:38.862 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T11:19:38.899 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T11:19:39.027 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:19:39.170 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:19:39.245 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T11:19:39.263 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T11:19:39.303 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:19:39.554 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T11:19:39.639 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-py. 2023-10-06T11:19:39.656 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T11:19:39.697 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:19:39.830 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:19:40.042 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T11:19:40.060 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T11:19:40.100 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:19:40.106 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:19:40.448 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T11:19:40.469 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T11:19:40.474 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:19:40.506 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T11:19:40.767 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T11:19:40.876 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T11:19:40.893 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T11:19:40.934 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:19:41.097 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T11:19:41.246 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T11:19:41.265 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T11:19:41.303 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:19:41.395 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T11:19:41.697 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:19:41.824 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:19:41.950 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:19:42.066 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package radosgw. 2023-10-06T11:19:42.086 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../114-radosgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:42.124 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:19:42.678 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:19:42.979 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:19:43.390 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:19:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T11:19:43.886 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T11:19:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:43.905 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:19:43.944 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:44.221 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:19:44.274 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T11:19:44.355 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T11:19:44.574 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:19:44.704 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:19:44.715 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:19:44.822 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:19:44.841 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:19:44.969 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T11:19:45.057 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:45.270 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:19:45.596 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:19:45.718 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user cephadm....done 2023-10-06T11:19:45.722 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:19:45.873 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:19:45.899 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:46.025 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:19:46.208 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:19:46.385 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T11:19:46.492 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T11:19:46.661 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:19:46.762 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T11:19:46.827 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:19:46.863 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:19:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:19:47.189 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T11:19:47.279 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T11:19:47.555 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:19:47.608 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:19:47.856 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:19:47.909 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T11:19:48.219 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T11:19:48.269 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T11:19:48.533 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:19:48.553 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:19:48.685 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:19:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:19:49.019 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:19:49.147 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:49.395 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:19:49.440 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:49.521 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:19:49.599 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T11:19:49.647 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T11:19:49.900 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:19:49.990 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:19:50.201 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:19:50.316 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librados2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:50.442 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T11:19:50.502 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:19:50.628 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:19:50.996 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T11:19:51.281 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:19:51.549 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:19:51.917 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:19:51.939 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:52.065 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:52.216 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T11:19:52.226 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T11:19:52.565 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T11:19:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T11:19:52.892 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T11:19:52.910 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:19:53.201 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:19:53.212 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:19:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T11:19:53.352 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:19:53.478 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:19:53.697 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:53.780 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:19:53.823 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:19:54.166 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:54.191 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:19:54.309 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:19:54.517 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:19:54.635 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:19:54.887 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:19:55.289 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T11:19:55.432 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librbd1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:55.808 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:19:55.858 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:19:55.942 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T11:19:55.992 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T11:19:56.119 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:19:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T11:19:56.338 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:19:56.466 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:19:56.546 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:19:56.623 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T11:19:56.680 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T11:19:56.916 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:19:57.058 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:57.218 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:57.284 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:19:57.435 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:19:57.451 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:19:57.451 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:19:57.561 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:19:57.896 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:19:57.902 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:58.020 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:19:58.197 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T11:19:58.321 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T11:19:58.499 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:19:58.624 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:19:58.639 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:19:58.959 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T11:19:59.024 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:59.160 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:59.252 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:19:59.294 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:59.412 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:59.538 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:19:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:19:59.664 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T11:19:59.959 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T11:19:59.973 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T11:20:00.231 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:20:00.267 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:00.382 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:20:00.387 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:00.513 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:20:00.691 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:20:01.068 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:20:01.219 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:20:01.345 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T11:20:01.600 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:01.680 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:20:02.014 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librados2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:02.140 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T11:20:02.153 INFO:teuthology.orchestra.run.smithi007.stdout:Adding group ceph....done 2023-10-06T11:20:02.529 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user ceph....done 2023-10-06T11:20:02.700 INFO:teuthology.orchestra.run.smithi007.stdout:Setting system user ceph properties....done 2023-10-06T11:20:02.726 INFO:teuthology.orchestra.run.smithi007.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T11:20:03.192 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T11:20:03.646 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:03.704 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:03.830 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:20:03.870 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:20:03.871 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:20:03.948 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T11:20:04.314 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:04.316 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T11:20:04.440 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:04.626 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:20:04.712 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:20:04.712 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:20:04.952 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:20:05.078 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T11:20:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:05.438 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:05.564 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:20:05.746 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T11:20:05.891 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:06.017 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:20:06.210 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:06.360 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:20:06.488 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:20:06.488 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:20:06.930 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:07.183 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:20:07.184 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:20:07.449 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librbd1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:07.575 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:20:07.630 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:07.701 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T11:20:07.860 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:20:07.986 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T11:20:08.133 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:20:08.133 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:20:08.396 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:20:08.566 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:08.692 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:08.738 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T11:20:08.835 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.063 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:20:09.064 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:20:09.117 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.243 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.473 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:20:09.473 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:20:09.595 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.738 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.864 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.944 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:10.079 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:20:10.092 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:20:10.246 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:20:10.372 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T11:20:10.522 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:20:10.699 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:20:11.093 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:11.227 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:11.353 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:11.480 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:11.631 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T11:20:12.045 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T11:20:12.354 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:12.522 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:12.648 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:12.774 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:20:13.566 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T11:20:13.753 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:20:14.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T11:20:14.180 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2023-10-06T11:20:14.339 INFO:teuthology.orchestra.run.smithi050.stdout:Adding group ceph....done 2023-10-06T11:20:14.389 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2023-10-06T11:20:14.390 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T11:20:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:20:14.604 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:20:14.604 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:20:14.604 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2023-10-06T11:20:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T11:20:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T11:20:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T11:20:14.605 INFO:teuthology.orchestra.run.smithi007.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T11:20:14.606 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2023-10-06T11:20:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T11:20:14.607 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T11:20:14.656 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2023-10-06T11:20:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T11:20:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T11:20:14.657 INFO:teuthology.orchestra.run.smithi007.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T11:20:14.658 INFO:teuthology.orchestra.run.smithi007.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T11:20:14.659 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2023-10-06T11:20:14.660 INFO:teuthology.orchestra.run.smithi007.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T11:20:14.717 INFO:teuthology.orchestra.run.smithi007.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-06T11:20:14.717 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 10.4 MB of archives. 2023-10-06T11:20:14.717 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T11:20:14.718 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T11:20:14.743 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user ceph....done 2023-10-06T11:20:14.847 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T11:20:14.847 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T11:20:14.848 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T11:20:14.852 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T11:20:14.853 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T11:20:14.871 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T11:20:14.871 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T11:20:14.873 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T11:20:14.874 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T11:20:14.874 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T11:20:14.880 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T11:20:14.881 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T11:20:14.882 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T11:20:14.882 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T11:20:14.896 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T11:20:14.897 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T11:20:14.902 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T11:20:14.913 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T11:20:14.913 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T11:20:14.927 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T11:20:14.927 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T11:20:14.934 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T11:20:14.944 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T11:20:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:Setting system user ceph properties....done 2023-10-06T11:20:14.959 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T11:20:14.962 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T11:20:14.962 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T11:20:14.963 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T11:20:14.963 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T11:20:14.968 INFO:teuthology.orchestra.run.smithi050.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T11:20:15.126 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:Preconfiguring packages ... 2023-10-06T11:20:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 10.4 MB in 0s (35.3 MB/s) 2023-10-06T11:20:15.443 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T11:20:15.637 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:15.642 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:15.907 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:16.137 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:20:16.137 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:20:16.592 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:16.718 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:16.979 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:16.993 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:20:16.993 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:20:17.645 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:18.061 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T11:20:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:18.514 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:18.551 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:18.739 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:20:18.739 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:20:18.974 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:19.124 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:19.185 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:19.417 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:20:19.418 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:20:19.462 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:19.911 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:20.188 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:20.193 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:20.350 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:20:20.395 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:20:20.395 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:20:20.806 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:20.838 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:20:20.964 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:21.107 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:21.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:20:21.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:20:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:20:21.585 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T11:20:21.816 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:21.984 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:22.110 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:22.131 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T11:20:22.149 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T11:20:22.190 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:20:22.253 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:20:22.363 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:20:22.531 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:20:22.551 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package dump. 2023-10-06T11:20:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T11:20:22.609 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T11:20:22.850 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:20:22.913 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package indent. 2023-10-06T11:20:22.934 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:20:22.971 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:20:23.258 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T11:20:23.276 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T11:20:23.307 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:20:23.628 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T11:20:23.645 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T11:20:23.686 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:20:23.989 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T11:20:24.007 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:24.047 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:24.359 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T11:20:24.377 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:24.409 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:24.703 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T11:20:24.723 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:24.761 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:25.081 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T11:20:25.099 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:25.247 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:25.794 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T11:20:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:20:26.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T11:20:26.489 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-10-06T11:20:26.696 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-10-06T11:20:26.697 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-10-06T11:20:26.733 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T11:20:26.754 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T11:20:26.903 INFO:teuthology.orchestra.run.smithi050.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T11:20:26.903 INFO:teuthology.orchestra.run.smithi050.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T11:20:26.903 INFO:teuthology.orchestra.run.smithi050.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T11:20:26.904 INFO:teuthology.orchestra.run.smithi050.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T11:20:26.904 INFO:teuthology.orchestra.run.smithi050.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T11:20:26.904 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:20:26.904 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:20:26.905 INFO:teuthology.orchestra.run.smithi050.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:20:26.905 INFO:teuthology.orchestra.run.smithi050.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:20:26.905 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2023-10-06T11:20:26.905 INFO:teuthology.orchestra.run.smithi050.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T11:20:26.905 INFO:teuthology.orchestra.run.smithi050.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T11:20:26.906 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T11:20:26.906 INFO:teuthology.orchestra.run.smithi050.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T11:20:26.906 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2023-10-06T11:20:26.906 INFO:teuthology.orchestra.run.smithi050.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T11:20:26.907 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T11:20:26.955 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2023-10-06T11:20:26.956 INFO:teuthology.orchestra.run.smithi050.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T11:20:26.956 INFO:teuthology.orchestra.run.smithi050.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T11:20:26.956 INFO:teuthology.orchestra.run.smithi050.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T11:20:26.957 INFO:teuthology.orchestra.run.smithi050.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T11:20:26.957 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2023-10-06T11:20:26.958 INFO:teuthology.orchestra.run.smithi050.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T11:20:27.143 INFO:teuthology.orchestra.run.smithi050.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-06T11:20:27.143 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 10.4 MB of archives. 2023-10-06T11:20:27.143 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T11:20:27.143 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T11:20:27.232 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T11:20:27.249 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:20:27.290 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:27.685 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T11:20:27.705 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:27.735 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:27.767 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T11:20:27.769 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T11:20:27.770 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T11:20:27.783 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T11:20:27.786 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T11:20:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T11:20:27.887 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T11:20:27.894 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T11:20:27.899 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T11:20:27.899 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T11:20:28.054 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T11:20:28.067 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T11:20:28.072 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:28.112 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:28.340 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T11:20:28.407 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T11:20:28.426 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:20:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:28.469 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T11:20:28.475 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T11:20:28.570 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T11:20:28.574 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T11:20:28.657 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T11:20:28.730 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T11:20:28.731 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T11:20:28.820 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T11:20:28.820 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T11:20:28.985 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T11:20:29.003 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T11:20:29.029 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T11:20:29.043 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:20:29.422 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package modemmanager. 2023-10-06T11:20:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:29.668 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T11:20:29.782 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T11:20:29.797 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T11:20:29.799 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T11:20:29.801 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T11:20:29.802 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T11:20:30.095 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T11:20:30.112 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T11:20:30.144 INFO:teuthology.orchestra.run.smithi050.stdout:Preconfiguring packages ... 2023-10-06T11:20:30.153 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:20:30.286 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 10.4 MB in 3s (3,677 kB/s) 2023-10-06T11:20:30.574 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:30.675 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package network-manager. 2023-10-06T11:20:30.694 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:20:30.758 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:30.772 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:31.425 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ppp. 2023-10-06T11:20:31.445 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T11:20:31.727 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:20:31.890 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:33.340 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T11:20:33.360 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T11:20:33.390 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T11:20:33.393 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:33.397 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:33.529 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:33.735 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T11:20:33.755 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T11:20:33.831 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T11:20:33.946 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:20:34.088 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T11:20:34.108 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T11:20:34.135 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:20:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T11:20:34.550 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T11:20:34.568 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T11:20:34.611 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:20:34.894 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:35.039 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:35.221 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:20:35.348 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:20:35.474 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:20:35.482 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152665 files and directories currently installed.) 2023-10-06T11:20:35.486 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:35.620 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:20:35.852 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:20:35.853 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:20:36.103 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:36.267 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:20:36.327 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:36.454 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:20:36.580 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:36.714 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T11:20:36.783 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:20:36.840 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:36.914 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T11:20:36.966 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:20:37.493 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T11:20:37.511 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T11:20:37.551 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:20:37.921 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package dump. 2023-10-06T11:20:37.931 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T11:20:37.939 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T11:20:37.979 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T11:20:38.164 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T11:20:38.276 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T11:20:38.316 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package indent. 2023-10-06T11:20:38.334 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:20:38.341 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:38.374 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:20:38.475 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:20:38.576 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:20:38.677 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:20:38.702 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T11:20:38.721 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T11:20:38.761 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:20:38.879 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:20:39.005 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:39.123 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T11:20:39.131 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:39.140 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T11:20:39.181 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:20:39.257 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T11:20:39.445 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:39.475 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T11:20:39.493 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:20:39.534 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:39.560 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:39.878 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T11:20:39.898 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:39.937 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout:which means they are currently configured by ifupdown: 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout:- ens1f0 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T11:20:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:20:40.427 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T11:20:40.746 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T11:20:40.783 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T11:20:40.801 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:40.841 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:41.194 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T11:20:41.212 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:41.252 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:41.283 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T11:20:41.639 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T11:20:41.656 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T11:20:41.697 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:20:41.771 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:41.906 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T11:20:42.049 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:42.058 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T11:20:42.076 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:20:42.117 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:42.175 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:42.326 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:42.452 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:42.528 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T11:20:42.545 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:42.586 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:42.720 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:20:42.721 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:20:42.956 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T11:20:42.976 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:43.014 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:43.222 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T11:20:43.283 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T11:20:43.303 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:20:43.341 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:43.410 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T11:20:43.854 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T11:20:43.874 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T11:20:43.882 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:20:43.912 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:20:44.033 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:20:44.299 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package modemmanager. 2023-10-06T11:20:44.317 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:20:44.327 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:20:44.359 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:44.988 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T11:20:45.009 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T11:20:45.046 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:20:45.595 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package network-manager. 2023-10-06T11:20:45.612 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:20:45.678 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:46.737 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ppp. 2023-10-06T11:20:46.755 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T11:20:47.035 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:20:47.367 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T11:20:47.386 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T11:20:47.459 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T11:20:47.845 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T11:20:47.866 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T11:20:47.921 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T11:20:48.165 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T11:20:48.182 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T11:20:48.259 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T11:20:48.296 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T11:20:48.404 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T11:20:48.677 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T11:20:48.697 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T11:20:48.745 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:20:49.054 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:49.374 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:20:49.533 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:20:49.614 INFO:teuthology.orchestra.run.smithi007.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:20:49.617 DEBUG:teuthology.parallel:result is None 2023-10-06T11:20:49.659 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:20:50.081 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:20:50.082 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:20:50.571 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:20:50.696 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:20:50.857 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:20:50.990 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T11:20:51.117 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:51.268 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:20:52.411 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T11:20:52.632 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T11:20:52.727 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T11:20:52.812 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:52.943 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:20:53.069 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:20:53.195 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:20:53.397 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:20:53.523 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:53.691 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:53.817 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T11:20:53.985 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:54.136 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout:which means they are currently configured by ifupdown: 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout:- ens1f0 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T11:20:54.837 INFO:teuthology.orchestra.run.smithi050.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T11:20:54.838 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:20:55.007 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T11:20:55.328 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T11:20:55.880 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T11:20:56.389 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:20:56.524 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T11:20:57.067 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:20:57.193 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:57.319 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:20:57.478 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:20:57.757 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:20:57.757 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:20:58.315 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T11:20:58.423 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T11:20:58.541 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:20:58.734 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:20:59.022 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:21:02.955 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T11:21:03.055 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T11:21:03.952 INFO:teuthology.orchestra.run.smithi050.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:21:03.954 DEBUG:teuthology.parallel:result is None 2023-10-06T11:21:03.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:21:04.102 DEBUG:teuthology.orchestra.run.smithi007:> dpkg-query -W -f '${Version}' ceph 2023-10-06T11:21:04.125 INFO:teuthology.orchestra.run.smithi007.stdout:17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:21:04.126 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:21:04.126 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274-g68e53a4a-1focal is installed. 2023-10-06T11:21:04.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:21:04.268 DEBUG:teuthology.orchestra.run.smithi050:> dpkg-query -W -f '${Version}' ceph 2023-10-06T11:21:04.292 INFO:teuthology.orchestra.run.smithi050.stdout:17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:21:04.292 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:21:04.292 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274-g68e53a4a-1focal is installed. 2023-10-06T11:21:04.293 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T11:21:04.293 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:04.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:21:04.309 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:04.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:21:04.351 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T11:21:04.351 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:04.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:21:04.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:21:04.385 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:04.385 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:21:04.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:21:04.462 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T11:21:04.462 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:04.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:21:04.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:21:04.534 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:04.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:21:04.551 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:21:04.567 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T11:21:04.567 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:04.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:21:04.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:21:04.592 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:04.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:21:04.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:21:04.637 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T11:21:04.697 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T11:21:04.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2023-10-06T11:21:04.699 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2023-10-06T11:21:04.715 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T11:21:04.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:21:04.717 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:21:04.773 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T11:21:04.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:21:04.776 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:21:04.795 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T11:21:04.795 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T11:21:04.795 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T11:21:04.795 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:04.795 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:21:04.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:04.875 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:04.875 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:21:04.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:04.968 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T11:21:04.968 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T11:21:04.968 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412395', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'snapdirname': '.customsnapkernel'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T11:21:04.969 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:21:04.971 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:21:04.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:21:05.021 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:21:05.036 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.036 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:21:05.041 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:21:05.041 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2023-10-06T11:21:05.048 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:21:05.048 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 357 Links: 1 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-06 11:16:37.864835893 +0000 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-06 11:16:37.584842346 +0000 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-06 11:16:37.584842346 +0000 2023-10-06T11:21:05.049 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-06T11:21:05.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-06T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-06T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00029247 s, 1.8 MB/s 2023-10-06T11:21:05.105 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:21:05.113 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 364 Links: 1 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-06 11:16:38.368824276 +0000 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-06 11:16:38.080830914 +0000 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-06 11:16:38.080830914 +0000 2023-10-06T11:21:05.121 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-06T11:21:05.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:21:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-06T11:21:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-06T11:21:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000221437 s, 2.3 MB/s 2023-10-06T11:21:05.137 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:21:05.145 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2023-10-06T11:21:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:21:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.194 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 372 Links: 1 2023-10-06T11:21:05.194 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.194 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-06 11:16:38.920811552 +0000 2023-10-06T11:21:05.195 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-06 11:16:38.596819021 +0000 2023-10-06T11:21:05.195 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-06 11:16:38.596819021 +0000 2023-10-06T11:21:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-06T11:21:05.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-06T11:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-06T11:21:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00029354 s, 1.7 MB/s 2023-10-06T11:21:05.208 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:21:05.216 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2023-10-06T11:21:05.266 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:21:05.266 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.266 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 375 Links: 1 2023-10-06T11:21:05.266 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.266 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-06 11:16:39.144806389 +0000 2023-10-06T11:21:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-06 11:16:39.140806481 +0000 2023-10-06T11:21:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-06 11:16:39.140806481 +0000 2023-10-06T11:21:05.267 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-06T11:21:05.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:21:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-06T11:21:05.282 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-06T11:21:05.282 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000295543 s, 1.7 MB/s 2023-10-06T11:21:05.283 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:21:05.290 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:05.290 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:05.290 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:21:05.297 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:21:05.297 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-06 11:16:38.116596621 +0000 2023-10-06T11:21:05.304 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-06 11:16:37.848602786 +0000 2023-10-06T11:21:05.305 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-06 11:16:37.848602786 +0000 2023-10-06T11:21:05.305 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-10-06T11:21:05.305 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:21:05.319 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-06T11:21:05.319 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-06T11:21:05.320 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000235933 s, 2.2 MB/s 2023-10-06T11:21:05.320 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:21:05.329 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-06 11:16:38.612585210 +0000 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-06 11:16:38.320591928 +0000 2023-10-06T11:21:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-06 11:16:38.320591928 +0000 2023-10-06T11:21:05.337 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-10-06T11:21:05.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:21:05.347 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-06T11:21:05.347 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-06T11:21:05.347 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000321661 s, 1.6 MB/s 2023-10-06T11:21:05.348 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:21:05.399 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2023-10-06T11:21:05.406 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 371 Links: 1 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-06 11:16:39.132573246 +0000 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-06 11:16:38.840579964 +0000 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-06 11:16:38.840579964 +0000 2023-10-06T11:21:05.407 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-10-06T11:21:05.407 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:21:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-06T11:21:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-06T11:21:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000304224 s, 1.7 MB/s 2023-10-06T11:21:05.422 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:21:05.471 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2023-10-06T11:21:05.478 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:21:05.478 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 380 Links: 1 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-06 11:16:39.644561467 +0000 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-06 11:16:39.340568460 +0000 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-06 11:16:39.340568460 +0000 2023-10-06T11:21:05.479 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-10-06T11:21:05.479 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:21:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-06T11:21:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-06T11:21:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000302208 s, 1.7 MB/s 2023-10-06T11:21:05.491 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:21:05.543 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:05.543 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:21:05.543 INFO:tasks.ceph:Generating config... 2023-10-06T11:21:05.544 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T11:21:05.544 INFO:tasks.ceph:[client] client snapdir = .customsnapfuse 2023-10-06T11:21:05.544 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T11:21:05.544 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T11:21:05.545 INFO:tasks.ceph:[client] pid file = 2023-10-06T11:21:05.545 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T11:21:05.545 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T11:21:05.545 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T11:21:05.546 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T11:21:05.546 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T11:21:05.546 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T11:21:05.547 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T11:21:05.547 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T11:21:05.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T11:21:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T11:21:05.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T11:21:05.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:21:05.634 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.50'), ('mon.c', '[v2:172.21.15.50:3301,v1:172.21.15.50:6790]')] 2023-10-06T11:21:05.634 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.50,[v2:172.21.15.50:3301,v1:172.21.15.50:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T11:21:05.635 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.635 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:21:05.682 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.50 --addv c '[v2:172.21.15.50:3301,v1:172.21.15.50:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:05.720 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:05.720 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = octopus 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2023-10-06T11:21:05.717517+0000 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:created 2023-10-06T11:21:05.717517+0000 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 15 (octopus) 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] mon.c 2023-10-06T11:21:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T11:21:05.723 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:21:05.727 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 88789558-6143-4e37-9ef9-c1260e585533... 2023-10-06T11:21:05.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:21:05.730 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:21:05.762 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2023-10-06T11:21:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2023-10-06T11:21:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2023-10-06T11:21:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:21:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2023-10-06T11:21:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.50,[v2:172.21.15.50:3301,v1:172.21.15.50:6790]" 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2023-10-06T11:21:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2023-10-06T11:21:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-06T11:21:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:21:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:21:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:05.772 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: client snapdir = .customsnapfuse 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-06T11:21:05.773 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2023-10-06T11:21:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2023-10-06T11:21:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-06T11:21:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-06T11:21:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-06T11:21:05.775 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.777 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:21:05.778 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:21:05.779 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "172.21.15.7,172.21.15.50,[v2:172.21.15.50:3301,v1:172.21.15.50:6790]" 2023-10-06T11:21:05.780 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.781 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2023-10-06T11:21:05.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 20 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 20 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2023-10-06T11:21:05.783 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 10 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-06T11:21:05.784 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:21:05.785 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:21:05.786 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: mon op complaint time = 120 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:21:05.787 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: client mount timeout = 600 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: client snapdir = .customsnapfuse 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = "" 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2023-10-06T11:21:05.788 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout:[mds] 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds = 20 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug frag = True 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug scatterstat = True 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: mds op complaint time = 180 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: mds verify scatter = True 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-06T11:21:05.789 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-06T11:21:05.790 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T11:21:05.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T11:21:05.879 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T11:21:05.879 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.880 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T11:21:05.887 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.887 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T11:21:05.893 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:21:05.893 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:21:05.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:21:05.962 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:05.962 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:05.967 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:21:05.967 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:05.967 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:21:05.967 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:21:05.995 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:05.995 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:06.045 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T11:21:06.045 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T11:21:06.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:21:06.092 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:21:06.095 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:21:06.142 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:21:06.144 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T11:21:06.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:21:06.190 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:21:06.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T11:21:06.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:21:06.291 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:21:06.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T11:21:06.308 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:21:06.357 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:21:06.359 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T11:21:06.370 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:21:06.413 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:21:06.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T11:21:06.431 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T11:21:06.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T11:21:06.466 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T11:21:06.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T11:21:06.512 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T11:21:06.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T11:21:06.560 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T11:21:06.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T11:21:06.609 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T11:21:06.621 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T11:21:06.622 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:21:06.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T11:21:06.638 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:06.639 INFO:tasks.ceph:role: osd.0 2023-10-06T11:21:06.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:21:06.639 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:21:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2023-10-06T11:21:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:06.725 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-06T11:21:06.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T11:21:06.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T11:21:06.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:06.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T11:21:06.766 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:06.766 INFO:tasks.ceph:role: osd.1 2023-10-06T11:21:06.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:21:06.767 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:21:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:06.808 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-06T11:21:06.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T11:21:06.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T11:21:06.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:06.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T11:21:06.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:06.895 INFO:tasks.ceph:role: osd.2 2023-10-06T11:21:06.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:21:06.895 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:21:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:06.936 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:06.937 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-06T11:21:06.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T11:21:06.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T11:21:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:07.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:07.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T11:21:07.020 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:07.020 INFO:tasks.ceph:role: osd.3 2023-10-06T11:21:07.020 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:21:07.020 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:07.062 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:07.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-06T11:21:07.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T11:21:07.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T11:21:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:07.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:07.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T11:21:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T11:21:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:21:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:21:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:21:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.194+0000 7f353dab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T11:21:07.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:07.758+0000 7f353dab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:08.534+0000 7f353dab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:09.330+0000 7f353dab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:10.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T11:21:10.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T11:21:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T11:21:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:21:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:21:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:21:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.266+0000 7f772bc993c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T11:21:10.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:10.846+0000 7f772bc993c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:11.630+0000 7f772bc993c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:12.418+0000 7f772bc993c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:13.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T11:21:13.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T11:21:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T11:21:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:21:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:21:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:21:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.382+0000 7f44e562f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T11:21:13.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:13.950+0000 7f44e562f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:14.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:14.730+0000 7f44e562f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:15.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:15.510+0000 7f44e562f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:16.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T11:21:16.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T11:21:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T11:21:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:21:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:21:16.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:21:16.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:16.478+0000 7f5ada4543c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T11:21:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:17.046+0000 7f5ada4543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:17.822+0000 7f5ada4543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:18.614+0000 7f5ada4543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:19.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T11:21:19.512 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T11:21:19.530 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:19.530 INFO:tasks.ceph:role: osd.4 2023-10-06T11:21:19.530 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:21:19.530 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:21:19.572 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:19.572 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:19.572 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:19.573 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:19.574 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-06T11:21:19.574 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T11:21:19.595 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T11:21:19.651 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:19.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:19.652 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T11:21:19.662 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:19.662 INFO:tasks.ceph:role: osd.5 2023-10-06T11:21:19.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:21:19.663 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:21:19.700 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:19.700 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:19.701 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:19.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-06T11:21:19.702 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T11:21:19.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T11:21:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:19.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:19.780 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T11:21:19.791 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:19.791 INFO:tasks.ceph:role: osd.6 2023-10-06T11:21:19.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:21:19.791 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:19.832 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:19.834 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-06T11:21:19.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T11:21:19.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T11:21:19.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:19.905 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:19.906 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T11:21:19.916 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:21:19.917 INFO:tasks.ceph:role: osd.7 2023-10-06T11:21:19.917 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:21:19.917 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:21:19.959 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:21:19.960 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:21:19.960 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:21:19.960 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:21:19.961 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-06T11:21:19.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T11:21:19.979 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T11:21:19.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:21:19.989 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:21:19.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:20.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.092+0000 7f57113673c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T11:21:20.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.092+0000 7f57113673c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T11:21:20.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.092+0000 7f57113673c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:21:20.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.092+0000 7f57113673c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:21:20.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.092+0000 7f57113673c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:21:20.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.096+0000 7f57113673c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T11:21:20.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:20.656+0000 7f57113673c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:21.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:21.436+0000 7f57113673c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:22.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:22.224+0000 7f57113673c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:23.083 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T11:21:23.099 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:23.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.128+0000 7fc729d473c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T11:21:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.128+0000 7fc729d473c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T11:21:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.128+0000 7fc729d473c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:21:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.132+0000 7fc729d473c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:21:23.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.132+0000 7fc729d473c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:21:23.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.132+0000 7fc729d473c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T11:21:23.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:23.688+0000 7fc729d473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:24.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:24.468+0000 7fc729d473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:25.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:25.248+0000 7fc729d473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:26.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T11:21:26.128 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T11:21:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T11:21:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:21:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:21:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:21:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.164+0000 7f58699bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T11:21:26.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:26.732+0000 7f58699bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:27.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:27.524+0000 7f58699bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:28.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:28.308+0000 7f58699bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:29.167 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T11:21:29.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:29.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T11:21:29.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T11:21:29.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:21:29.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:21:29.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:21:29.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.264+0000 7ff0006283c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T11:21:29.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:29.820+0000 7ff0006283c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:30.616+0000 7ff0006283c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:31.408+0000 7ff0006283c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:21:32.267 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T11:21:32.283 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T11:21:32.284 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T11:21:32.301 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T11:21:32.360 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T11:21:32.370 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T11:21:32.380 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T11:21:32.395 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T11:21:32.409 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T11:21:32.467 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T11:21:32.484 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T11:21:32.543 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T11:21:32.552 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T11:21:32.568 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.568 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T11:21:32.630 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T11:21:32.640 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.640 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T11:21:32.656 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.656 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T11:21:32.663 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:32.663 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T11:21:32.670 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.670 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T11:21:32.705 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:32.705 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T11:21:32.712 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T11:21:32.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:21:32.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:21:32.727 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2023-10-06T11:21:32.727 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9ldAdbBRAA8GCg4bKcrzuzkaduCi8XvA== 2023-10-06T11:21:32.727 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9l1+wnCxAAWCyrtRm4/l01bVOfl3RFNA== 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lXLcqERAANaP6kbnJHMe9OCl2sjMa1g== 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCj7R9lGH/WCxAAzuwYCO4wsYR2zGf1W29XZw== 2023-10-06T11:21:32.728 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCm7R9lY1MzEBAAlxX4RI/KFowzyFqUncP6xQ== 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCp7R9lJykPFxAAxwaaC+xoN9U+SHpI2iz1/Q== 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCs7R9l1vLTHBAAxImlU3qarF5ZOKn+wiKYMA== 2023-10-06T11:21:32.729 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2023-10-06T11:21:32.730 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lXbVmCBAAPUux0zkxVGM3gHdQhuk4VQ== 2023-10-06T11:21:32.730 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-10-06T11:21:32.730 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9laHI5FRAA3af4MNcnGb9VDo5Ijo30SA== 2023-10-06T11:21:32.730 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-10-06T11:21:32.730 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lVyWRGBAA5iEiW4W9odLEUorf3t5EPQ== 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCw7R9lVEbJBRAAj48N2fdwZO74MPfqZrBgLQ== 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCz7R9l0YT3BxAAh/FAxhszZOD8HHADQWv86Q== 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2023-10-06T11:21:32.731 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQC27R9lwmb3CRAA7kBK8youRc70G4y9C7DZnA== 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQC57R9l9PngDxAAiYqjQtUm9kSgphe0jP8Jeg== 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lAsmfGxAA/rcxLJ7QHrdvJ+qQEx6gOw== 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2023-10-06T11:21:32.732 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lAXRWHhAAzypkRTEsOQwk3ogrI6HrMQ== 2023-10-06T11:21:32.733 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2023-10-06T11:21:32.733 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lip1bIRAAuhH8cb2ryJKFjGCWT7lUVg== 2023-10-06T11:21:32.733 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2023-10-06T11:21:32.733 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCi7R9lov88JBAAR9JDw9oMtv3OKzAZ5F+kiw== 2023-10-06T11:21:32.735 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.y] 2023-10-06T11:21:32.735 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9ldAdbBRAA8GCg4bKcrzuzkaduCi8XvA== 2023-10-06T11:21:32.735 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2023-10-06T11:21:32.735 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9l1+wnCxAAWCyrtRm4/l01bVOfl3RFNA== 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lXLcqERAANaP6kbnJHMe9OCl2sjMa1g== 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCj7R9lGH/WCxAAzuwYCO4wsYR2zGf1W29XZw== 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2023-10-06T11:21:32.736 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCm7R9lY1MzEBAAlxX4RI/KFowzyFqUncP6xQ== 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCp7R9lJykPFxAAxwaaC+xoN9U+SHpI2iz1/Q== 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.3] 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCs7R9l1vLTHBAAxImlU3qarF5ZOKn+wiKYMA== 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2023-10-06T11:21:32.737 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lXbVmCBAAPUux0zkxVGM3gHdQhuk4VQ== 2023-10-06T11:21:32.738 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2023-10-06T11:21:32.738 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9laHI5FRAA3af4MNcnGb9VDo5Ijo30SA== 2023-10-06T11:21:32.738 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2023-10-06T11:21:32.738 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lVyWRGBAA5iEiW4W9odLEUorf3t5EPQ== 2023-10-06T11:21:32.738 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.4] 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCw7R9lVEbJBRAAj48N2fdwZO74MPfqZrBgLQ== 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.5] 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCz7R9l0YT3BxAAh/FAxhszZOD8HHADQWv86Q== 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.6] 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQC27R9lwmb3CRAA7kBK8youRc70G4y9C7DZnA== 2023-10-06T11:21:32.739 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.7] 2023-10-06T11:21:32.740 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQC57R9l9PngDxAAiYqjQtUm9kSgphe0jP8Jeg== 2023-10-06T11:21:32.740 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2023-10-06T11:21:32.740 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lAsmfGxAA/rcxLJ7QHrdvJ+qQEx6gOw== 2023-10-06T11:21:32.740 INFO:teuthology.orchestra.run.smithi050.stdout:[client.2] 2023-10-06T11:21:32.740 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lAXRWHhAAzypkRTEsOQwk3ogrI6HrMQ== 2023-10-06T11:21:32.741 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2023-10-06T11:21:32.741 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lip1bIRAAuhH8cb2ryJKFjGCWT7lUVg== 2023-10-06T11:21:32.741 INFO:teuthology.orchestra.run.smithi050.stdout:[client.3] 2023-10-06T11:21:32.741 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCi7R9lov88JBAAR9JDw9oMtv3OKzAZ5F+kiw== 2023-10-06T11:21:32.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:21:32.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:21:32.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:32.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:32.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:32.879 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:32.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:32.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:32.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:32.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.053 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:21:33.155 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:21:33.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:33.234 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:33.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:33.283 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:21:33.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.332 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.417 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:21:33.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.652 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.699 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:21:33.796 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T11:21:33.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T11:21:33.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:21:33.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T11:21:33.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T11:21:33.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:21:33.951 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T11:21:33.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T11:21:34.020 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:21:34.079 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T11:21:34.092 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:34.094 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:21:34.098 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T11:21:34.099 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T11:21:34.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T11:21:34.101 INFO:tasks.ceph.mon.a:Started 2023-10-06T11:21:34.101 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T11:21:34.102 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T11:21:34.140 INFO:tasks.ceph.mon.b:Started 2023-10-06T11:21:34.140 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T11:21:34.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T11:21:34.143 INFO:tasks.ceph.mon.c:Started 2023-10-06T11:21:34.143 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T11:21:34.143 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T11:21:34.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T11:21:34.146 INFO:tasks.ceph.mgr.y:Started 2023-10-06T11:21:34.146 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T11:21:34.146 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T11:21:34.148 INFO:tasks.ceph.mgr.x:Started 2023-10-06T11:21:34.149 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T11:21:34.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T11:21:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.242+0000 7f0909ca0700 1 Processor -- start 2023-10-06T11:21:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.242+0000 7f0909ca0700 1 -- start start 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0909ca0700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0909ca0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090415b6e0 con 0x7f0904156d80 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0909ca0700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090406e780 con 0x7f09040c9240 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f08fbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0904156d80 0x7f09040c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41222/0 (socket says 172.21.15.7:41222) 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 learned_addr learned my addr 172.21.15.7:0/4253615202 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0904066c00 con 0x7f09041569b0 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:34.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:34.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:21:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.450+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.450+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.450+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.450+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.450+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:21:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.854+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.854+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.854+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.854+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:34.854+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:21:34.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:34.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:35.658+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:35.658+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:35.658+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:35.658+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:35.658+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:21:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 msgr2=0x7f090406ec10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f09041569b0 0x7f090406ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 >> v1:172.21.15.50:6789/0 conn(0x7f09040c9240 legacy=0x7f090415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 >> v1:172.21.15.7:6789/0 conn(0x7f0904156d80 legacy=0x7f09040c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f0008d30 con 0x7f08f0003710 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0008ea0 con 0x7f08f00061c0 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.246+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:21:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.446+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:37.850+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:21:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:21:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f0908c9e700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f08fbfff700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:21:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:38.650+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:21:39.729 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:39.723+0000 7f6095474000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 msgr2=0x7f08f0005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f0003710 0x7f08f0005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 -- 172.21.15.7:0/4253615202 reap_dead start 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f08f0003b10 0x7f08f00092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f00097e0 0x7f08f000bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f0008d30 con 0x7f08f00097e0 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f0008ea0 con 0x7f08f0003b10 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fa7fc700 1 -- 172.21.15.7:0/4253615202 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f000c260 con 0x7f08f00061c0 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.7:3300/0 conn(0x7f08f00097e0 0x7f08f000bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f0908c9e700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fbfff700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f08f0003b10 0x7f08f00092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 msgr2=0x7f08f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 -- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f08f0003b10 msgr2=0x7f08f00092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f08f0003b10 0x7f08f00092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 -- 172.21.15.7:0/4253615202 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090415b400 con 0x7f08f00097e0 2023-10-06T11:21:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f090949f700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00097e0 0x7f08f000bbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09040cd200 tx=0x7f08fc0016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7346e0f297d3ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fb7fe700 1 -- 172.21.15.7:0/4253615202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005da0 con 0x7f08f00097e0 2023-10-06T11:21:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fb7fe700 1 -- 172.21.15.7:0/4253615202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc0070d0 con 0x7f08f00097e0 2023-10-06T11:21:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.246+0000 7f08fb7fe700 1 -- 172.21.15.7:0/4253615202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0074d0 con 0x7f08f00097e0 2023-10-06T11:21:40.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00097e0 msgr2=0x7f08f000bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00097e0 0x7f08f000bbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09040cd200 tx=0x7f08fc0016c0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 shutdown_connections 2023-10-06T11:21:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00097e0 0x7f08f000bbc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 --2- 172.21.15.7:0/4253615202 >> v2:172.21.15.50:3300/0 conn(0x7f08f0003b10 0x7f08f00092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 --2- 172.21.15.7:0/4253615202 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f00061c0 0x7f08f0008630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 >> 172.21.15.7:0/4253615202 conn(0x7f0904067e30 msgr2=0x7f0904068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 shutdown_connections 2023-10-06T11:21:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.250+0000 7f0909ca0700 1 -- 172.21.15.7:0/4253615202 wait complete. 2023-10-06T11:21:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 Processor -- start 2023-10-06T11:21:40.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 -- start start 2023-10-06T11:21:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f0003b10 0x7f09040ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00061c0 0x7f09040cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c4de0 con 0x7f08f00061c0 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cac50 con 0x7f08f00097e0 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.254+0000 7f0909ca0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040caf20 con 0x7f08f0003b10 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f08fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00061c0 0x7f09040cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f0908c9e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f0003b10 0x7f09040ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:56644/0 (socket says 172.21.15.7:56644) 2023-10-06T11:21:40.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 -- 172.21.15.7:0/2530766661 learned_addr learned my addr 172.21.15.7:0/2530766661 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:40.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 -- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f0003b10 msgr2=0x7f09040ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f0003b10 0x7f09040ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 -- 172.21.15.7:0/2530766661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00061c0 msgr2=0x7f09040cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00061c0 0x7f09040cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 -- 172.21.15.7:0/2530766661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090406e780 con 0x7f08f00097e0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f090949f700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f090406e6f0 tx=0x7f08fc015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc007d70 con 0x7f08f00097e0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc013a10 con 0x7f08f00097e0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09040c4790 con 0x7f08f00097e0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc013e10 con 0x7f08f00097e0 2023-10-06T11:21:40.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.258+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040ccb50 con 0x7f08f00097e0 2023-10-06T11:21:40.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.262+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f08fc01e680 con 0x7f08f00097e0 2023-10-06T11:21:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.262+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f08fc0287b0 con 0x7f08f00097e0 2023-10-06T11:21:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.262+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040c4f50 con 0x7f08f00097e0 2023-10-06T11:21:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.266+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f08fc01eac0 con 0x7f08f00097e0 2023-10-06T11:21:40.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.338+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f090406e130 con 0x7f08f00097e0 2023-10-06T11:21:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f08f8ff9700 1 -- 172.21.15.7:0/2530766661 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f08fc01ec60 con 0x7f08f00097e0 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 msgr2=0x7f09040cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f090406e6f0 tx=0x7f08fc015000 comp rx=0 tx=0).stop 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 shutdown_connections 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08f00097e0 0x7f09040cb430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f00061c0 0x7f09040cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 --2- 172.21.15.7:0/2530766661 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f08f0003b10 0x7f09040ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 >> 172.21.15.7:0/2530766661 conn(0x7f0904067e30 msgr2=0x7f09040ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 shutdown_connections 2023-10-06T11:21:40.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:40.346+0000 7f0909ca0700 1 -- 172.21.15.7:0/2530766661 wait complete. 2023-10-06T11:21:40.355 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:40.347+0000 7f6095474000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:21:40.355 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2023-10-06T11:21:40.366 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T11:21:40.367 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:40.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T11:21:40.376 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:40.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T11:21:40.412 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:40.407+0000 7f6095474000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:21:40.433 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:40.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T11:21:40.436 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:40.430+0000 7fb2241b5000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:21:40.444 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-06T11:21:40.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T11:21:40.454 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:40.454 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T11:21:40.470 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:40.470 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T11:21:40.485 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:40.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T11:21:40.494 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-06T11:21:40.494 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T11:21:40.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 7b851f7e-bc0d-49e3-966e-45a3192e5cdb 0 2023-10-06T11:21:40.531 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:40.527+0000 7f6095474000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:21:40.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 Processor -- start 2023-10-06T11:21:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 -- start start 2023-10-06T11:21:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febd406e110 0x7febd4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7febd4155d10 0x7febd40c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd4156620 con 0x7febd4155940 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd4159ff0 con 0x7febd406e110 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febdbb4f700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd415a2d0 con 0x7febd4155d10 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd98eb700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7febd4155940 0x7febd41597a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44278/0 (socket says 172.21.15.50:44278) 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd98eb700 1 -- 172.21.15.50:0/2692731892 learned_addr learned my addr 172.21.15.50:0/2692731892 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 --2- 172.21.15.50:0/2692731892 >> v2:172.21.15.50:3300/0 conn(0x7febd4155d10 0x7febd40c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd90ea700 1 --2- 172.21.15.50:0/2692731892 >> v2:172.21.15.7:3300/0 conn(0x7febd406e110 0x7febd4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd88e9700 1 -- 172.21.15.50:0/2692731892 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628154398 0 0) 0x7febd4156620 con 0x7febd4155940 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd88e9700 1 -- 172.21.15.50:0/2692731892 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febbc003570 con 0x7febd4155940 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 -- 172.21.15.50:0/2692731892 >> v2:172.21.15.7:3300/0 conn(0x7febd406e110 msgr2=0x7febd4156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 --2- 172.21.15.50:0/2692731892 >> v2:172.21.15.7:3300/0 conn(0x7febd406e110 0x7febd4156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 -- 172.21.15.50:0/2692731892 >> v1:172.21.15.50:6789/0 conn(0x7febd4155940 legacy=0x7febd41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 -- 172.21.15.50:0/2692731892 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd415a5b0 con 0x7febd4155d10 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febda0ec700 1 --2- 172.21.15.50:0/2692731892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd4155d10 0x7febd40c7f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febd40634b0 tx=0x7febcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2577709c73274c78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.631+0000 7febd88e9700 1 -- 172.21.15.50:0/2692731892 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011630 con 0x7febd4155d10 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febd88e9700 1 -- 172.21.15.50:0/2692731892 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc0117b0 con 0x7febd4155d10 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febd88e9700 1 -- 172.21.15.50:0/2692731892 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011b30 con 0x7febd4155d10 2023-10-06T11:21:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- 172.21.15.50:0/2692731892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd4155d10 msgr2=0x7febd40c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 --2- 172.21.15.50:0/2692731892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd4155d10 0x7febd40c7f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febd40634b0 tx=0x7febcc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- 172.21.15.50:0/2692731892 shutdown_connections 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 --2- 172.21.15.50:0/2692731892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd4155d10 0x7febd40c7f10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 --2- 172.21.15.50:0/2692731892 >> v2:172.21.15.7:3300/0 conn(0x7febd406e110 0x7febd4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- 172.21.15.50:0/2692731892 >> 172.21.15.50:0/2692731892 conn(0x7febd40cc2f0 msgr2=0x7febd40cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- 172.21.15.50:0/2692731892 shutdown_connections 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- 172.21.15.50:0/2692731892 wait complete. 2023-10-06T11:21:40.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 Processor -- start 2023-10-06T11:21:40.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.635+0000 7febdbb4f700 1 -- start start 2023-10-06T11:21:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 0x7febd40afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 0x7febd4156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4155d10 0x7febd414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4156e90 con 0x7febd4155d10 2023-10-06T11:21:40.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41570c0 con 0x7febd406e110 2023-10-06T11:21:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd41573a0 con 0x7febd4155940 2023-10-06T11:21:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd90ea700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 0x7febd4156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 0x7febd40afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd90ea700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 0x7febd4156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38790/0 (socket says 172.21.15.50:38790) 2023-10-06T11:21:40.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd90ea700 1 -- 172.21.15.50:0/3232868244 learned_addr learned my addr 172.21.15.50:0/3232868244 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febda0ec700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4155d10 0x7febd414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 -- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 msgr2=0x7febd4156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 0x7febd4156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 -- 172.21.15.50:0/3232868244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4155d10 msgr2=0x7febd414e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4155d10 0x7febd414e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 -- 172.21.15.50:0/3232868244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009420 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd98eb700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 0x7febd40afbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febc400efd0 tx=0x7febc4001ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4013070 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc400e2e0 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4157680 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4010be0 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4157c40 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7febc400c6d0 con 0x7febd406e110 2023-10-06T11:21:40.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7febc4022900 con 0x7febd406e110 2023-10-06T11:21:40.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.639+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd4157e60 con 0x7febd406e110 2023-10-06T11:21:40.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.643+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7febc400cb10 con 0x7febd406e110 2023-10-06T11:21:40.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.719+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b851f7e-bc0d-49e3-966e-45a3192e5cdb", "id": 0} v 0) v1 -- 0x7febd404be00 con 0x7febd406e110 2023-10-06T11:21:40.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febd1ffb700 1 -- 172.21.15.50:0/3232868244 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b851f7e-bc0d-49e3-966e-45a3192e5cdb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7febc400ccb0 con 0x7febd406e110 2023-10-06T11:21:40.731 INFO:teuthology.orchestra.run.smithi050.stdout:0 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 msgr2=0x7febd40afbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 0x7febd40afbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7febc400efd0 tx=0x7febc4001ad0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 shutdown_connections 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4155d10 0x7febd414e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd4155940 0x7febd4156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 --2- 172.21.15.50:0/3232868244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd406e110 0x7febd40afbc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 >> 172.21.15.50:0/3232868244 conn(0x7febd40cc2f0 msgr2=0x7febd414f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 shutdown_connections 2023-10-06T11:21:40.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.727+0000 7febdbb4f700 1 -- 172.21.15.50:0/3232868244 wait complete. 2023-10-06T11:21:40.746 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new bbf26255-a037-41f7-a74f-fa7274055679 1 2023-10-06T11:21:40.782 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:40.775+0000 7f6095474000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:21:40.836 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:40.831+0000 7f6095474000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:21:40.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.839+0000 7f656b612700 1 Processor -- start 2023-10-06T11:21:40.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.839+0000 7f656b612700 1 -- start start 2023-10-06T11:21:40.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f656406a4f0 0x7f65640c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656406e900 con 0x7f656406f2d0 2023-10-06T11:21:40.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656415a4b0 con 0x7f656406a4f0 2023-10-06T11:21:40.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656415a1d0 con 0x7f6564156810 2023-10-06T11:21:40.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38804/0 (socket says 172.21.15.50:38804) 2023-10-06T11:21:40.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 -- 172.21.15.50:0/4070175683 learned_addr learned my addr 172.21.15.50:0/4070175683 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:40.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f6569baf700 1 --2- 172.21.15.50:0/4070175683 >> v2:172.21.15.7:3300/0 conn(0x7f656406a4f0 0x7f65640c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 -- 172.21.15.50:0/4070175683 >> v2:172.21.15.7:3300/0 conn(0x7f656406a4f0 msgr2=0x7f65640c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:40.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 --2- 172.21.15.50:0/4070175683 >> v2:172.21.15.7:3300/0 conn(0x7f656406a4f0 0x7f65640c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 -- 172.21.15.50:0/4070175683 >> v1:172.21.15.7:6789/0 conn(0x7f656406f2d0 legacy=0x7f6564069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 -- 172.21.15.50:0/4070175683 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656406e2b0 con 0x7f6564156810 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f65693ae700 1 --2- 172.21.15.50:0/4070175683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65600099b0 tx=0x7f6560009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcb51da13a8d6d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f655bfff700 1 -- 172.21.15.50:0/4070175683 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560009ea0 con 0x7f6564156810 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f655bfff700 1 -- 172.21.15.50:0/4070175683 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000b860 con 0x7f6564156810 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f655bfff700 1 -- 172.21.15.50:0/4070175683 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000bc00 con 0x7f6564156810 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- 172.21.15.50:0/4070175683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 msgr2=0x7f656406ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 --2- 172.21.15.50:0/4070175683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65600099b0 tx=0x7f6560009250 comp rx=0 tx=0).stop 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- 172.21.15.50:0/4070175683 shutdown_connections 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 --2- 172.21.15.50:0/4070175683 >> v2:172.21.15.7:3300/0 conn(0x7f656406a4f0 0x7f65640c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 --2- 172.21.15.50:0/4070175683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6564156810 0x7f656406ed90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- 172.21.15.50:0/4070175683 >> 172.21.15.50:0/4070175683 conn(0x7f65640cc2f0 msgr2=0x7f65640cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- 172.21.15.50:0/4070175683 shutdown_connections 2023-10-06T11:21:40.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.843+0000 7f656b612700 1 -- 172.21.15.50:0/4070175683 wait complete. 2023-10-06T11:21:40.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 Processor -- start 2023-10-06T11:21:40.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 -- start start 2023-10-06T11:21:40.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 0x7f6564154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6564156810 0x7f656406b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c5820 con 0x7f656406a4f0 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c4ca0 con 0x7f6564156810 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f656b612700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640c4f80 con 0x7f656406f2d0 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38818/0 (socket says 172.21.15.50:38818) 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 -- 172.21.15.50:0/2837219189 learned_addr learned my addr 172.21.15.50:0/2837219189 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6569baf700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6564156810 0x7f656406b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f65693ae700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 0x7f6564154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:40.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 -- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6564156810 msgr2=0x7f656406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6564156810 0x7f656406b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 -- 172.21.15.50:0/2837219189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 msgr2=0x7f6564154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 0x7f6564154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f6568bad700 1 -- 172.21.15.50:0/2837219189 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600094f0 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.847+0000 7f65693ae700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 0x7f6564154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f6568bad700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f655400c590 tx=0x7f655400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554011590 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554011710 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65640c5260 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554011ad0 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6564154770 con 0x7f656406f2d0 2023-10-06T11:21:40.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f655401c6c0 con 0x7f656406f2d0 2023-10-06T11:21:40.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f65540127e0 con 0x7f656406f2d0 2023-10-06T11:21:40.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.851+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65640af220 con 0x7f656406f2d0 2023-10-06T11:21:40.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.855+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f65640af220 con 0x7f656406f2d0 2023-10-06T11:21:40.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.927+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bbf26255-a037-41f7-a74f-fa7274055679", "id": 1} v 0) v1 -- 0x7f656404be00 con 0x7f656406f2d0 2023-10-06T11:21:40.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f65597fa700 1 -- 172.21.15.50:0/2837219189 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bbf26255-a037-41f7-a74f-fa7274055679", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6554017070 con 0x7f656406f2d0 2023-10-06T11:21:40.941 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 msgr2=0x7f65640c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f655400c590 tx=0x7f655400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 shutdown_connections 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6564156810 0x7f656406b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f656406f2d0 0x7f65640c4400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.935+0000 7f656b612700 1 --2- 172.21.15.50:0/2837219189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f656406a4f0 0x7f6564154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.939+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 >> 172.21.15.50:0/2837219189 conn(0x7f65640cc2f0 msgr2=0x7f65640c2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:40.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.939+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 shutdown_connections 2023-10-06T11:21:40.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:40.939+0000 7f656b612700 1 -- 172.21.15.50:0/2837219189 wait complete. 2023-10-06T11:21:40.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new ab91019f-a131-455c-9f84-862984c7ee84 2 2023-10-06T11:21:41.031 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:41.027+0000 7f6095474000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:21:41.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.051+0000 7f592860a700 1 Processor -- start 2023-10-06T11:21:41.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.051+0000 7f592860a700 1 -- start start 2023-10-06T11:21:41.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f59201567f0 0x7f59201561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 0x7f5920156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59200add80 con 0x7f59201597a0 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5920159f80 con 0x7f59201567f0 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592015a260 con 0x7f592006ef50 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f59201567f0 0x7f59201561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f59201567f0 0x7f59201561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40850/0 (socket says 172.21.15.50:40850) 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 -- 172.21.15.50:0/524903764 learned_addr learned my addr 172.21.15.50:0/524903764 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f5925ba5700 1 --1- 172.21.15.50:0/524903764 >> v1:172.21.15.50:6789/0 conn(0x7f59201597a0 0x7f592006e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44294/0 (socket says 172.21.15.50:44294) 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f5926ba7700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 0x7f5920156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 -- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 msgr2=0x7f5920156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 0x7f5920156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 -- 172.21.15.50:0/524903764 >> v1:172.21.15.50:6789/0 conn(0x7f59201597a0 legacy=0x7f592006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 -- 172.21.15.50:0/524903764 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592015a540 con 0x7f59201567f0 2023-10-06T11:21:41.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59253a4700 1 -- 172.21.15.50:0/524903764 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707372738 0 0) 0x7f59200add80 con 0x7f59201597a0 2023-10-06T11:21:41.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f5926ba7700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 0x7f5920156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:41.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59263a6700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59201567f0 0x7f59201561a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f591c00bc50 tx=0x7f591c00b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f17a023d3c88104 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59253a4700 1 -- 172.21.15.50:0/524903764 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00dbf0 con 0x7f59201567f0 2023-10-06T11:21:41.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59253a4700 1 -- 172.21.15.50:0/524903764 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c00dd50 con 0x7f59201567f0 2023-10-06T11:21:41.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f59253a4700 1 -- 172.21.15.50:0/524903764 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0196b0 con 0x7f59201567f0 2023-10-06T11:21:41.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59201567f0 msgr2=0x7f59201561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59201567f0 0x7f59201561a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f591c00bc50 tx=0x7f591c00b4f0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- 172.21.15.50:0/524903764 shutdown_connections 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f592006ef50 0x7f5920156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 --2- 172.21.15.50:0/524903764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59201567f0 0x7f59201561a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- 172.21.15.50:0/524903764 >> 172.21.15.50:0/524903764 conn(0x7f5920066840 msgr2=0x7f5920066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- 172.21.15.50:0/524903764 shutdown_connections 2023-10-06T11:21:41.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.055+0000 7f592860a700 1 -- 172.21.15.50:0/524903764 wait complete. 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 Processor -- start 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- start start 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 0x7f59200b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59201597a0 0x7f5920152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c8d00 con 0x7f59201597a0 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200a94a0 con 0x7f592006ef50 2023-10-06T11:21:41.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c8200 con 0x7f59201567f0 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59263a6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 0x7f59200b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38836/0 (socket says 172.21.15.50:38836) 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 -- 172.21.15.50:0/1923579477 learned_addr learned my addr 172.21.15.50:0/1923579477 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59263a6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 0x7f59200b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40852/0 (socket says 172.21.15.50:40852) 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5926ba7700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59201597a0 0x7f5920152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 -- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 msgr2=0x7f59200b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 0x7f59200b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 -- 172.21.15.50:0/1923579477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59201597a0 msgr2=0x7f5920152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59201597a0 0x7f5920152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 -- 172.21.15.50:0/1923579477 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5918009a20 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f5925ba5700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59100011e0 tx=0x7f5910007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910005a70 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5910005bd0 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c00b790 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100070d0 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200c87c0 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f591001c440 con 0x7f59201567f0 2023-10-06T11:21:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.059+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f5910012830 con 0x7f59201567f0 2023-10-06T11:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.063+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592004be00 con 0x7f59201567f0 2023-10-06T11:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.063+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5910017070 con 0x7f59201567f0 2023-10-06T11:21:41.094 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:41.087+0000 7f6095474000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:21:41.094 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.086+0000 7fb2241b5000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:21:41.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.135+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ab91019f-a131-455c-9f84-862984c7ee84", "id": 2} v 0) v1 -- 0x7f59200a95e0 con 0x7f59201567f0 2023-10-06T11:21:41.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.143+0000 7f59167fc700 1 -- 172.21.15.50:0/1923579477 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab91019f-a131-455c-9f84-862984c7ee84", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f591001c8d0 con 0x7f59201567f0 2023-10-06T11:21:41.150 INFO:teuthology.orchestra.run.smithi050.stdout:2 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.143+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 msgr2=0x7f59200c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59100011e0 tx=0x7f5910007e10 comp rx=0 tx=0).stop 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 shutdown_connections 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59201597a0 0x7f5920152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f59201567f0 0x7f59200c7cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 --2- 172.21.15.50:0/1923579477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f592006ef50 0x7f59200b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 >> 172.21.15.50:0/1923579477 conn(0x7f5920066840 msgr2=0x7f59200ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 shutdown_connections 2023-10-06T11:21:41.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.147+0000 7f592860a700 1 -- 172.21.15.50:0/1923579477 wait complete. 2023-10-06T11:21:41.152 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.146+0000 7fb2241b5000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:21:41.157 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:41.151+0000 7f6095474000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:21:41.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 399b2d85-efb6-4838-93ed-3010a09e786e 3 2023-10-06T11:21:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 Processor -- start 2023-10-06T11:21:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 -- start start 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 0x7faaf00c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faaf006bfb0 0x7faaf00c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf00c9a60 con 0x7faaf0155c30 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00caea0 con 0x7faaf0155860 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00c9c90 con 0x7faaf006bfb0 2023-10-06T11:21:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faaf006bfb0 0x7faaf00c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faaf006bfb0 0x7faaf00c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40864/0 (socket says 172.21.15.50:40864) 2023-10-06T11:21:41.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 0x7faaf00c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 -- 172.21.15.50:0/1728642647 learned_addr learned my addr 172.21.15.50:0/1728642647 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.271 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.262+0000 7fb2241b5000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:21:41.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 -- 172.21.15.50:0/1728642647 >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 msgr2=0x7faaf00c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 --2- 172.21.15.50:0/1728642647 >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 0x7faaf00c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 -- 172.21.15.50:0/1728642647 >> v1:172.21.15.7:6789/0 conn(0x7faaf0155c30 legacy=0x7faaf006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaedd9b700 1 -- 172.21.15.50:0/1728642647 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155660859 0 0) 0x7faaf00c9a60 con 0x7faaf0155c30 2023-10-06T11:21:41.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 -- 172.21.15.50:0/1728642647 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00ca570 con 0x7faaf006bfb0 2023-10-06T11:21:41.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaee59c700 1 --2- 172.21.15.50:0/1728642647 >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 0x7faaf00c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaeed9d700 1 --2- 172.21.15.50:0/1728642647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf006bfb0 0x7faaf00c9520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faaf00af460 tx=0x7faae8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d2b3b5e8e1d697 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaedd9b700 1 -- 172.21.15.50:0/1728642647 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800e070 con 0x7faaf006bfb0 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaedd9b700 1 -- 172.21.15.50:0/1728642647 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae800ba20 con 0x7faaf006bfb0 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaedd9b700 1 -- 172.21.15.50:0/1728642647 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80092e0 con 0x7faaf006bfb0 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 -- 172.21.15.50:0/1728642647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf006bfb0 msgr2=0x7faaf00c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.263+0000 7faaef59e700 1 --2- 172.21.15.50:0/1728642647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf006bfb0 0x7faaf00c9520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faaf00af460 tx=0x7faae8009840 comp rx=0 tx=0).stop 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1728642647 shutdown_connections 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 --2- 172.21.15.50:0/1728642647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf006bfb0 0x7faaf00c9520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 --2- 172.21.15.50:0/1728642647 >> v2:172.21.15.7:3300/0 conn(0x7faaf0155860 0x7faaf00c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1728642647 >> 172.21.15.50:0/1728642647 conn(0x7faaf006f2a0 msgr2=0x7faaf0068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1728642647 shutdown_connections 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1728642647 wait complete. 2023-10-06T11:21:41.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 Processor -- start 2023-10-06T11:21:41.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- start start 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faaf006bfb0 0x7faaf00c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 0x7faaf01404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00c64b0 con 0x7faaf0155c30 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00af740 con 0x7faaf0155860 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00b02c0 con 0x7faaf006bfb0 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaee59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faaf006bfb0 0x7faaf00c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40874/0 (socket says 172.21.15.50:40874) 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 -- 172.21.15.50:0/1687232753 learned_addr learned my addr 172.21.15.50:0/1687232753 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaeed9d700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 0x7faaf01404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 -- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faaf006bfb0 msgr2=0x7faaf00c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faaf006bfb0 0x7faaf00c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 -- 172.21.15.50:0/1687232753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 msgr2=0x7faaf01404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 0x7faaf01404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 -- 172.21.15.50:0/1687232753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009420 con 0x7faaf0155860 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaeed9d700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 0x7faaf01404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae5d9b700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faad800c530 tx=0x7faad800c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8011590 con 0x7faaf0155860 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8011710 con 0x7faaf0155860 2023-10-06T11:21:41.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae8009ae0 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8011ad0 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.267+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf00aff50 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.271+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faad801c6c0 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.271+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7faad8001010 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.271+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf006da60 con 0x7faaf0155860 2023-10-06T11:21:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.271+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faad8017070 con 0x7faaf0155860 2023-10-06T11:21:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.347+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd new", "uuid": "399b2d85-efb6-4838-93ed-3010a09e786e", "id": 3} v 0) v1 -- 0x7faaf004b5c0 con 0x7faaf0155860 2023-10-06T11:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faae6ffd700 1 -- 172.21.15.50:0/1687232753 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "399b2d85-efb6-4838-93ed-3010a09e786e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7faad801ccd0 con 0x7faaf0155860 2023-10-06T11:21:41.359 INFO:teuthology.orchestra.run.smithi050.stdout:3 2023-10-06T11:21:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 msgr2=0x7faaf0157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faad800c530 tx=0x7faad800c840 comp rx=0 tx=0).stop 2023-10-06T11:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 shutdown_connections 2023-10-06T11:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaf0155c30 0x7faaf01404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faaf0155860 0x7faaf0157130 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 --2- 172.21.15.50:0/1687232753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faaf006bfb0 0x7faaf00c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 >> 172.21.15.50:0/1687232753 conn(0x7faaf006f2a0 msgr2=0x7faaf00c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 shutdown_connections 2023-10-06T11:21:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.355+0000 7faaef59e700 1 -- 172.21.15.50:0/1687232753 wait complete. 2023-10-06T11:21:41.372 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new ea94ba68-eeed-4571-b2fc-3d175337222f 4 2023-10-06T11:21:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 Processor -- start 2023-10-06T11:21:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 -- start start 2023-10-06T11:21:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78a01561a0 0x7f78a006e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a006f1e0 con 0x7f78a006ed80 2023-10-06T11:21:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a00add80 con 0x7f78a01567f0 2023-10-06T11:21:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78a586c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a0159f80 con 0x7f78a01561a0 2023-10-06T11:21:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789effd700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f78a01567f0 0x7f78a01597a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44304/0 (socket says 172.21.15.50:44304) 2023-10-06T11:21:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789effd700 1 -- 172.21.15.50:0/2053284661 learned_addr learned my addr 172.21.15.50:0/2053284661 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-06T11:21:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78967fc700 1 --2- 172.21.15.50:0/2053284661 >> v2:172.21.15.50:3300/0 conn(0x7f78a01561a0 0x7f78a006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789e7fc700 1 -- 172.21.15.50:0/2053284661 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902102976 0 0) 0x7f78a00add80 con 0x7f78a01567f0 2023-10-06T11:21:41.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789e7fc700 1 -- 172.21.15.50:0/2053284661 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f788c003570 con 0x7f78a01567f0 2023-10-06T11:21:41.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78967fc700 1 -- 172.21.15.50:0/2053284661 >> v1:172.21.15.50:6789/0 conn(0x7f78a01567f0 legacy=0x7f78a01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78967fc700 1 -- 172.21.15.50:0/2053284661 >> v1:172.21.15.7:6789/0 conn(0x7f78a006ed80 legacy=0x7f78a0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78967fc700 1 -- 172.21.15.50:0/2053284661 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a015a260 con 0x7f78a01561a0 2023-10-06T11:21:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f78967fc700 1 --2- 172.21.15.50:0/2053284661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01561a0 0x7f78a006e840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f78880098e0 tx=0x7f78880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1059b7c6a32184 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789e7fc700 1 -- 172.21.15.50:0/2053284661 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788800b640 con 0x7f78a01561a0 2023-10-06T11:21:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789e7fc700 1 -- 172.21.15.50:0/2053284661 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788800b7a0 con 0x7f78a01561a0 2023-10-06T11:21:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.475+0000 7f789e7fc700 1 -- 172.21.15.50:0/2053284661 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788800bb20 con 0x7f78a01561a0 2023-10-06T11:21:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- 172.21.15.50:0/2053284661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01561a0 msgr2=0x7f78a006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 --2- 172.21.15.50:0/2053284661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01561a0 0x7f78a006e840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f78880098e0 tx=0x7f78880091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- 172.21.15.50:0/2053284661 shutdown_connections 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 --2- 172.21.15.50:0/2053284661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01561a0 0x7f78a006e840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- 172.21.15.50:0/2053284661 >> 172.21.15.50:0/2053284661 conn(0x7f78a0066840 msgr2=0x7f78a0066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- 172.21.15.50:0/2053284661 shutdown_connections 2023-10-06T11:21:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- 172.21.15.50:0/2053284661 wait complete. 2023-10-06T11:21:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 Processor -- start 2023-10-06T11:21:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.479+0000 7f78a586c700 1 -- start start 2023-10-06T11:21:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a01561a0 0x7f78a0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01567f0 0x7f78a0150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00c43f0 con 0x7f78a01561a0 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a01573f0 con 0x7f78a01567f0 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a01576d0 con 0x7f78a006ed80 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01567f0 0x7f78a0150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38858/0 (socket says 172.21.15.50:38858) 2023-10-06T11:21:41.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 -- 172.21.15.50:0/594791468 learned_addr learned my addr 172.21.15.50:0/594791468 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78967fc700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a01561a0 0x7f78a0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 -- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01567f0 msgr2=0x7f78a0150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01567f0 0x7f78a0150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 -- 172.21.15.50:0/594791468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a01561a0 msgr2=0x7f78a0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a01561a0 0x7f78a0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 -- 172.21.15.50:0/594791468 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888009420 con 0x7f78a006ed80 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f789effd700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f789000c7f0 tx=0x7f7890007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890005cd0 con 0x7f78a006ed80 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7890005e50 con 0x7f78a006ed80 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890007320 con 0x7f78a006ed80 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a01579b0 con 0x7f78a006ed80 2023-10-06T11:21:41.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a0151230 con 0x7f78a006ed80 2023-10-06T11:21:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f789001c6c0 con 0x7f78a006ed80 2023-10-06T11:21:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f7890007720 con 0x7f78a006ed80 2023-10-06T11:21:41.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.483+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a0157bf0 con 0x7f78a006ed80 2023-10-06T11:21:41.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.491+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f78a0157bf0 con 0x7f78a006ed80 2023-10-06T11:21:41.515 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.506+0000 7fb2241b5000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:21:41.569 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.562+0000 7fb2241b5000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:21:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.563+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ea94ba68-eeed-4571-b2fc-3d175337222f", "id": 4} v 0) v1 -- 0x7f78a00c4150 con 0x7f78a006ed80 2023-10-06T11:21:41.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.571+0000 7f78977fe700 1 -- 172.21.15.50:0/594791468 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea94ba68-eeed-4571-b2fc-3d175337222f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f78a00c4150 con 0x7f78a006ed80 2023-10-06T11:21:41.578 INFO:teuthology.orchestra.run.smithi050.stdout:4 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 msgr2=0x7f78a0158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f789000c7f0 tx=0x7f7890007e00 comp rx=0 tx=0).stop 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 shutdown_connections 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78a01567f0 0x7f78a0150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a01561a0 0x7f78a0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 --2- 172.21.15.50:0/594791468 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78a006ed80 0x7f78a0158230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 >> 172.21.15.50:0/594791468 conn(0x7f78a0066840 msgr2=0x7f78a00aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 shutdown_connections 2023-10-06T11:21:41.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f78a586c700 1 -- 172.21.15.50:0/594791468 wait complete. 2023-10-06T11:21:41.581 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:41.575+0000 7f6095474000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:21:41.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new ddad3fd1-81da-47a1-8d35-5c213d4051da 5 2023-10-06T11:21:41.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.679+0000 7fe9cd45a700 1 Processor -- start 2023-10-06T11:21:41.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.679+0000 7fe9cd45a700 1 -- start start 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9c806c2e0 0x7fe9c8163d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80c8be0 con 0x7fe9c815c8a0 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c806aa70 con 0x7fe9c815c4d0 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c806ad50 con 0x7fe9c806c2e0 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c6ffd700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe9c815c4d0 0x7fe9c8069ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44308/0 (socket says 172.21.15.50:44308) 2023-10-06T11:21:41.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c6ffd700 1 -- 172.21.15.50:0/1165183942 learned_addr learned my addr 172.21.15.50:0/1165183942 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c77fe700 1 --2- 172.21.15.50:0/1165183942 >> v2:172.21.15.7:3300/0 conn(0x7fe9c806c2e0 0x7fe9c8163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336641253 0 0) 0x7fe9c806aa70 con 0x7fe9c815c4d0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b4003570 con 0x7fe9c815c4d0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.1 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3116977515 0 0) 0x7fe9b4003570 con 0x7fe9c815c4d0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 >> v2:172.21.15.7:3300/0 conn(0x7fe9c806c2e0 msgr2=0x7fe9c8163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 --2- 172.21.15.50:0/1165183942 >> v2:172.21.15.7:3300/0 conn(0x7fe9c806c2e0 0x7fe9c8163d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 >> v1:172.21.15.7:6789/0 conn(0x7fe9c815c8a0 legacy=0x7fe9c806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80c8870 con 0x7fe9c815c4d0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.1 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7fe9b8001470 con 0x7fe9c815c4d0 2023-10-06T11:21:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 >> v1:172.21.15.50:6789/0 conn(0x7fe9c815c4d0 legacy=0x7fe9c8069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b4003a60 0x7fe9b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b40051c0 0x7fe9b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806ad50 con 0x7fe9b40051c0 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b400c160 con 0x7fe9b4003a60 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b400c320 con 0x7fe9b4004810 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c6ffd700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b4003a60 0x7fe9b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c77fe700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b40051c0 0x7fe9b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 -- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b4003a60 msgr2=0x7fe9b4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b4003a60 0x7fe9b4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 -- 172.21.15.50:0/1165183942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b40051c0 msgr2=0x7fe9b400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b40051c0 0x7fe9b400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c77fe700 1 -- 172.21.15.50:0/1165183942 reap_dead start 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 -- 172.21.15.50:0/1165183942 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4003570 con 0x7fe9b4004810 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c6ffd700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b4003a60 0x7fe9b4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9bffff700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9b4004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00018b0 tx=0x7fe9b0007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5153c33961aad9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0012420 con 0x7fe9b4004810 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00125a0 con 0x7fe9b4004810 2023-10-06T11:21:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9c67fc700 1 -- 172.21.15.50:0/1165183942 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0012960 con 0x7fe9b4004810 2023-10-06T11:21:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 msgr2=0x7fe9b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9b4004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00018b0 tx=0x7fe9b0007d30 comp rx=0 tx=0).stop 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- 172.21.15.50:0/1165183942 shutdown_connections 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/1165183942 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9b4004c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.683+0000 7fe9cd45a700 1 -- 172.21.15.50:0/1165183942 >> 172.21.15.50:0/1165183942 conn(0x7fe9c80cdc90 msgr2=0x7fe9c80ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- 172.21.15.50:0/1165183942 shutdown_connections 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- 172.21.15.50:0/1165183942 wait complete. 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 Processor -- start 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- start start 2023-10-06T11:21:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 0x7fe9c80c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 0x7fe9c816cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b0590 con 0x7fe9c8155c50 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b0d70 con 0x7fe9c80ca480 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9cd45a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b31d0 con 0x7fe9b4004810 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.687+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38866/0 (socket says 172.21.15.50:38866) 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 -- 172.21.15.50:0/3006859936 learned_addr learned my addr 172.21.15.50:0/3006859936 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c77fe700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 0x7fe9c816cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9bffff700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 0x7fe9c80c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 -- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 msgr2=0x7fe9c816cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 0x7fe9c816cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 -- 172.21.15.50:0/3006859936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 msgr2=0x7fe9c80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 0x7fe9c80c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 -- 172.21.15.50:0/3006859936 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80ca8f0 con 0x7fe9b4004810 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c77fe700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 0x7fe9c816cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9bffff700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 0x7fe9c80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9c6ffd700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8007740 tx=0x7fe9b8001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8018030 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8001430 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80b39c0 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800fe90 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80b3400 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe9b801b070 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fe9b8019070 con 0x7fe9b4004810 2023-10-06T11:21:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.691+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c804be00 con 0x7fe9b4004810 2023-10-06T11:21:41.701 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:41.695+0000 7f6095474000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:21:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.695+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe9b8007770 con 0x7fe9b4004810 2023-10-06T11:21:41.766 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.758+0000 7fb2241b5000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:21:41.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.767+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ddad3fd1-81da-47a1-8d35-5c213d4051da", "id": 5} v 0) v1 -- 0x7fe9c80c4dd0 con 0x7fe9b4004810 2023-10-06T11:21:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9beffd700 1 -- 172.21.15.50:0/3006859936 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddad3fd1-81da-47a1-8d35-5c213d4051da", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe9b802d0b0 con 0x7fe9b4004810 2023-10-06T11:21:41.781 INFO:teuthology.orchestra.run.smithi050.stdout:5 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 msgr2=0x7fe9c80c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8007740 tx=0x7fe9b8001e50 comp rx=0 tx=0).stop 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 shutdown_connections 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c80ca480 0x7fe9c816cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c8155c50 0x7fe9c80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 --2- 172.21.15.50:0/3006859936 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe9b4004810 0x7fe9c80c7580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 >> 172.21.15.50:0/3006859936 conn(0x7fe9c80cdc90 msgr2=0x7fe9c80ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 shutdown_connections 2023-10-06T11:21:41.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.775+0000 7fe9cd45a700 1 -- 172.21.15.50:0/3006859936 wait complete. 2023-10-06T11:21:41.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 9f702162-6deb-48cb-8a8b-d1518c0b3420 6 2023-10-06T11:21:41.833 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.826+0000 7fb2241b5000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:21:41.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 Processor -- start 2023-10-06T11:21:41.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 -- start start 2023-10-06T11:21:41.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae8815a780 0x7fae8815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8815b6e0 con 0x7fae88156b40 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8806e780 con 0x7fae8815a780 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae8cabc700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8806f320 con 0x7fae880c8b30 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38868/0 (socket says 172.21.15.50:38868) 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 -- 172.21.15.50:0/3000911009 learned_addr learned my addr 172.21.15.50:0/3000911009 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae87fff700 1 --2- 172.21.15.50:0/3000911009 >> v2:172.21.15.7:3300/0 conn(0x7fae8815a780 0x7fae8815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 -- 172.21.15.50:0/3000911009 >> v2:172.21.15.7:3300/0 conn(0x7fae8815a780 msgr2=0x7fae8815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 --2- 172.21.15.50:0/3000911009 >> v2:172.21.15.7:3300/0 conn(0x7fae8815a780 0x7fae8815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 -- 172.21.15.50:0/3000911009 >> v1:172.21.15.7:6789/0 conn(0x7fae88156b40 legacy=0x7fae8806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 -- 172.21.15.50:0/3000911009 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae880c94c0 con 0x7fae880c8b30 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae86ffd700 1 --2- 172.21.15.50:0/3000911009 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0099b0 tx=0x7fae7c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e342cfb6e19e2980 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae867fc700 1 -- 172.21.15.50:0/3000911009 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00b860 con 0x7fae880c8b30 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae867fc700 1 -- 172.21.15.50:0/3000911009 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c00b9c0 con 0x7fae880c8b30 2023-10-06T11:21:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.891+0000 7fae867fc700 1 -- 172.21.15.50:0/3000911009 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00bd60 con 0x7fae880c8b30 2023-10-06T11:21:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- 172.21.15.50:0/3000911009 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 msgr2=0x7fae880c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3000911009 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0099b0 tx=0x7fae7c009250 comp rx=0 tx=0).stop 2023-10-06T11:21:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- 172.21.15.50:0/3000911009 shutdown_connections 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3000911009 >> v2:172.21.15.7:3300/0 conn(0x7fae8815a780 0x7fae8815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3000911009 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae880c8f80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- 172.21.15.50:0/3000911009 >> 172.21.15.50:0/3000911009 conn(0x7fae88067e30 msgr2=0x7fae88068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- 172.21.15.50:0/3000911009 shutdown_connections 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- 172.21.15.50:0/3000911009 wait complete. 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 Processor -- start 2023-10-06T11:21:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- start start 2023-10-06T11:21:41.903 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:41.894+0000 7fb2241b5000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae881596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae88156b40 0x7fae88148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8815a280 con 0x7fae88156b40 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae880c41b0 con 0x7fae8815a780 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae8cabc700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae880c35a0 con 0x7fae880c8b30 2023-10-06T11:21:41.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae877fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae881596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae877fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae881596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38884/0 (socket says 172.21.15.50:38884) 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40918/0 (socket says 172.21.15.50:40918) 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae877fe700 1 -- 172.21.15.50:0/3317497168 learned_addr learned my addr 172.21.15.50:0/3317497168 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae86ffd700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae88156b40 0x7fae88148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 -- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 msgr2=0x7fae881596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae881596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 -- 172.21.15.50:0/3317497168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae88156b40 msgr2=0x7fae88148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae88156b40 0x7fae88148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.895+0000 7fae87fff700 1 -- 172.21.15.50:0/3317497168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0094f0 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae87fff700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae80007370 tx=0x7fae800098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80015070 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae800100d0 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae880c3880 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000e450 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae880c3e40 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fae8000eb00 con 0x7fae8815a780 2023-10-06T11:21:41.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fae80019dc0 con 0x7fae8815a780 2023-10-06T11:21:41.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.899+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8806cf60 con 0x7fae8815a780 2023-10-06T11:21:41.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.903+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fae80027060 con 0x7fae8815a780 2023-10-06T11:21:41.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.975+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9f702162-6deb-48cb-8a8b-d1518c0b3420", "id": 6} v 0) v1 -- 0x7fae8804b5c0 con 0x7fae8815a780 2023-10-06T11:21:41.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.983+0000 7fae6f7fe700 1 -- 172.21.15.50:0/3317497168 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f702162-6deb-48cb-8a8b-d1518c0b3420", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae8000edb0 con 0x7fae8815a780 2023-10-06T11:21:41.990 INFO:teuthology.orchestra.run.smithi050.stdout:6 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.983+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 msgr2=0x7fae880ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.983+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae80007370 tx=0x7fae800098e0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 shutdown_connections 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae8815a780 0x7fae880ad770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae88156b40 0x7fae88148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 --2- 172.21.15.50:0/3317497168 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fae880c8b30 0x7fae881596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:41.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 >> 172.21.15.50:0/3317497168 conn(0x7fae88067e30 msgr2=0x7fae880b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:41.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 shutdown_connections 2023-10-06T11:21:41.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:41.987+0000 7fae8cabc700 1 -- 172.21.15.50:0/3317497168 wait complete. 2023-10-06T11:21:42.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 9b5682fa-9148-4801-9c00-e59ff2f62347 7 2023-10-06T11:21:42.008 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.003+0000 7f6095474000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:21:42.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.103+0000 7ff39f59e700 1 Processor -- start 2023-10-06T11:21:42.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.103+0000 7ff39f59e700 1 -- start start 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3a0069cd0 0x7ff3a006a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a006a650 con 0x7ff3a00c7c40 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a01570e0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00c8090 con 0x7ff3a0153620 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38886/0 (socket says 172.21.15.50:38886) 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 -- 172.21.15.50:0/2073726842 learned_addr learned my addr 172.21.15.50:0/2073726842 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:42.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39ed9d700 1 --2- 172.21.15.50:0/2073726842 >> v2:172.21.15.7:3300/0 conn(0x7ff3a0069cd0 0x7ff3a006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 -- 172.21.15.50:0/2073726842 >> v2:172.21.15.7:3300/0 conn(0x7ff3a0069cd0 msgr2=0x7ff3a006a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 --2- 172.21.15.50:0/2073726842 >> v2:172.21.15.7:3300/0 conn(0x7ff3a0069cd0 0x7ff3a006a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 -- 172.21.15.50:0/2073726842 >> v1:172.21.15.7:6789/0 conn(0x7ff3a00c7c40 legacy=0x7ff3a01569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 -- 172.21.15.50:0/2073726842 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00c8320 con 0x7ff3a0153620 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39e59c700 1 --2- 172.21.15.50:0/2073726842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d6dd15253dc6b3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39d59a700 1 -- 172.21.15.50:0/2073726842 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390009ea0 con 0x7ff3a0153620 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39d59a700 1 -- 172.21.15.50:0/2073726842 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39000b860 con 0x7ff3a0153620 2023-10-06T11:21:42.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39d59a700 1 -- 172.21.15.50:0/2073726842 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000bc00 con 0x7ff3a0153620 2023-10-06T11:21:42.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- 172.21.15.50:0/2073726842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 msgr2=0x7ff3a00ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 --2- 172.21.15.50:0/2073726842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390009250 comp rx=0 tx=0).stop 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- 172.21.15.50:0/2073726842 shutdown_connections 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 --2- 172.21.15.50:0/2073726842 >> v2:172.21.15.7:3300/0 conn(0x7ff3a0069cd0 0x7ff3a006a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 --2- 172.21.15.50:0/2073726842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0153620 0x7ff3a00ae5f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- 172.21.15.50:0/2073726842 >> 172.21.15.50:0/2073726842 conn(0x7ff3a0067e30 msgr2=0x7ff3a0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- 172.21.15.50:0/2073726842 shutdown_connections 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.107+0000 7ff39f59e700 1 -- 172.21.15.50:0/2073726842 wait complete. 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 Processor -- start 2023-10-06T11:21:42.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- start start 2023-10-06T11:21:42.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a00c7c40 0x7ff3a00aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0153620 0x7ff3a014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b0320 con 0x7ff3a00c7c40 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b0600 con 0x7ff3a0153620 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b08e0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.50:38892/0 (socket says 172.21.15.50:38892) 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 -- 172.21.15.50:0/65350653 learned_addr learned my addr 172.21.15.50:0/65350653 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39ed9d700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0153620 0x7ff3a014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39dd9b700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a00c7c40 0x7ff3a00aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 -- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0153620 msgr2=0x7ff3a014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0153620 0x7ff3a014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 -- 172.21.15.50:0/65350653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a00c7c40 msgr2=0x7ff3a00aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a00c7c40 0x7ff3a00aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 -- 172.21.15.50:0/65350653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900094f0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39e59c700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c81d0 tx=0x7ff390017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000bbe0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39001a450 con 0x7ff3a0069cd0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00b0bc0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39001a810 con 0x7ff3a0069cd0 2023-10-06T11:21:42.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a0168d60 con 0x7ff3a0069cd0 2023-10-06T11:21:42.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff39001abc0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.111+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7ff390011a90 con 0x7ff3a0069cd0 2023-10-06T11:21:42.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.115+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a006dfe0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.119+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff3a006dfe0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.179 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.175+0000 7f6095474000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:21:42.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.191+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9b5682fa-9148-4801-9c00-e59ff2f62347", "id": 7} v 0) v1 -- 0x7ff3a004b5c0 con 0x7ff3a0069cd0 2023-10-06T11:21:42.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff38e7fc700 1 -- 172.21.15.50:0/65350653 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b5682fa-9148-4801-9c00-e59ff2f62347", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff390029070 con 0x7ff3a0069cd0 2023-10-06T11:21:42.203 INFO:teuthology.orchestra.run.smithi050.stdout:7 2023-10-06T11:21:42.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 msgr2=0x7ff3a0168820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c81d0 tx=0x7ff390017000 comp rx=0 tx=0).stop 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 shutdown_connections 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3a0153620 0x7ff3a014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a00c7c40 0x7ff3a00aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 --2- 172.21.15.50:0/65350653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff3a0069cd0 0x7ff3a0168820 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 >> 172.21.15.50:0/65350653 conn(0x7ff3a0067e30 msgr2=0x7ff3a00caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 shutdown_connections 2023-10-06T11:21:42.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-06T11:21:42.199+0000 7ff39f59e700 1 -- 172.21.15.50:0/65350653 wait complete. 2023-10-06T11:21:42.215 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T11:21:42.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T11:21:42.218 INFO:tasks.ceph.osd.0:Started 2023-10-06T11:21:42.218 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T11:21:42.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T11:21:42.220 INFO:tasks.ceph.osd.1:Started 2023-10-06T11:21:42.221 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T11:21:42.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T11:21:42.223 INFO:tasks.ceph.osd.2:Started 2023-10-06T11:21:42.224 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T11:21:42.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T11:21:42.226 INFO:tasks.ceph.osd.3:Started 2023-10-06T11:21:42.226 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T11:21:42.227 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T11:21:42.229 INFO:tasks.ceph.osd.4:Started 2023-10-06T11:21:42.229 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T11:21:42.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T11:21:42.231 INFO:tasks.ceph.osd.5:Started 2023-10-06T11:21:42.231 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T11:21:42.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T11:21:42.234 INFO:tasks.ceph.osd.6:Started 2023-10-06T11:21:42.234 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T11:21:42.235 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T11:21:42.237 INFO:tasks.ceph.osd.7:Started 2023-10-06T11:21:42.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:21:42.249 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.243+0000 7f6095474000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:21:42.319 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.315+0000 7f6095474000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:21:42.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 Processor -- start 2023-10-06T11:21:42.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 -- start start 2023-10-06T11:21:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 0x7f85b80af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b80b04f0 con 0x7f85b815c170 2023-10-06T11:21:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b80af8e0 con 0x7f85b815c980 2023-10-06T11:21:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80afbc0 con 0x7f85b815c540 2023-10-06T11:21:42.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85b815c170 0x7f85b81593d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60394/0 (socket says 172.21.15.7:60394) 2023-10-06T11:21:42.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.366+0000 7f85b659c700 1 -- 172.21.15.7:0/2575666503 learned_addr learned my addr 172.21.15.7:0/2575666503 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:42.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b5d9b700 1 --2- 172.21.15.7:0/2575666503 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 0x7f85b80af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397333317 0 0) 0x7f85b80b04f0 con 0x7f85b815c170 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a0003570 con 0x7f85b815c170 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b5d9b700 1 -- 172.21.15.7:0/2575666503 >> v1:172.21.15.50:6789/0 conn(0x7f85b815c980 legacy=0x7f85b80b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559932442 0 0) 0x7f85b80af8e0 con 0x7f85b815c980 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b5d9b700 1 -- 172.21.15.7:0/2575666503 >> v1:172.21.15.7:6789/0 conn(0x7f85b815c170 legacy=0x7f85b81593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b5d9b700 1 -- 172.21.15.7:0/2575666503 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80afea0 con 0x7f85b815c540 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b5d9b700 1 --2- 172.21.15.7:0/2575666503 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 0x7f85b80af2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009910 tx=0x7f85ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3148b30e72a5254 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00b680 con 0x7f85b815c540 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00b7e0 con 0x7f85b815c540 2023-10-06T11:21:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b559a700 1 -- 172.21.15.7:0/2575666503 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00bba0 con 0x7f85b815c540 2023-10-06T11:21:42.378 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.371+0000 7f6095474000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- 172.21.15.7:0/2575666503 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 msgr2=0x7f85b80af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 --2- 172.21.15.7:0/2575666503 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 0x7f85b80af2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009910 tx=0x7f85ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- 172.21.15.7:0/2575666503 shutdown_connections 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 --2- 172.21.15.7:0/2575666503 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c540 0x7f85b80af2d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- 172.21.15.7:0/2575666503 >> 172.21.15.7:0/2575666503 conn(0x7f85b8068900 msgr2=0x7f85b80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- 172.21.15.7:0/2575666503 shutdown_connections 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- 172.21.15.7:0/2575666503 wait complete. 2023-10-06T11:21:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 Processor -- start 2023-10-06T11:21:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.370+0000 7f85b759e700 1 -- start start 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c170 0x7f85b80aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a9dd0 con 0x7f85b815c980 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a8c20 con 0x7f85b815c540 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a8f00 con 0x7f85b815c170 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b659c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c170 0x7f85b80aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:44008/0 (socket says 172.21.15.7:44008) 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40274/0 (socket says 172.21.15.7:40274) 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 -- 172.21.15.7:0/762483131 learned_addr learned my addr 172.21.15.7:0/762483131 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 -- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c170 msgr2=0x7f85b80aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c170 0x7f85b80aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 -- 172.21.15.7:0/762483131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 msgr2=0x7f85b80b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 -- 172.21.15.7:0/762483131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac009420 con 0x7f85b815c540 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b6d9d700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b5d9b700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f85b806ef80 tx=0x7f85ac017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00bb60 con 0x7f85b815c540 2023-10-06T11:21:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac000c90 con 0x7f85b815c540 2023-10-06T11:21:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b80a91e0 con 0x7f85b815c540 2023-10-06T11:21:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac027680 con 0x7f85b815c540 2023-10-06T11:21:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b80a97a0 con 0x7f85b815c540 2023-10-06T11:21:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f85ac027c70 con 0x7f85b815c540 2023-10-06T11:21:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.374+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f85ac01a430 con 0x7f85b815c540 2023-10-06T11:21:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.378+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b806c710 con 0x7f85b815c540 2023-10-06T11:21:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.382+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f85ac015020 con 0x7f85b815c540 2023-10-06T11:21:42.388 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:42.382+0000 7fb2241b5000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:21:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.446+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85b80b0180 con 0x7f85b815c540 2023-10-06T11:21:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.446+0000 7f85a67fc700 1 -- 172.21.15.7:0/762483131 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f85ac035090 con 0x7f85b815c540 2023-10-06T11:21:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:42.454 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:21:42.194948+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.446+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 msgr2=0x7f85b8157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.446+0000 7f85b759e700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f85b806ef80 tx=0x7f85ac017000 comp rx=0 tx=0).stop 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 shutdown_connections 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b815c980 0x7f85b80b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b815c540 0x7f85b8157af0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 --2- 172.21.15.7:0/762483131 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85b815c170 0x7f85b80aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 >> 172.21.15.7:0/762483131 conn(0x7f85b8068900 msgr2=0x7f85b815ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 shutdown_connections 2023-10-06T11:21:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:42.450+0000 7f85b759e700 1 -- 172.21.15.7:0/762483131 wait complete. 2023-10-06T11:21:42.472 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T11:21:42.473 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T11:21:42.513 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:42.506+0000 7fb2241b5000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:21:42.597 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-10-06T11:21:42.591+0000 7fba05c613c0 -1 Falling back to public interface 2023-10-06T11:21:42.597 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-06T11:21:42.590+0000 7fd79a2ab3c0 -1 Falling back to public interface 2023-10-06T11:21:42.601 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-06T11:21:42.594+0000 7fd86b63b3c0 -1 Falling back to public interface 2023-10-06T11:21:42.612 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-06T11:21:42.606+0000 7f5eb5ecc3c0 -1 Falling back to public interface 2023-10-06T11:21:42.616 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-10-06T11:21:42.611+0000 7f04d3a5a3c0 -1 Falling back to public interface 2023-10-06T11:21:42.628 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-06T11:21:42.622+0000 7ff11cc3f3c0 -1 Falling back to public interface 2023-10-06T11:21:42.676 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.671+0000 7f6095474000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:21:42.734 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.727+0000 7f6095474000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:21:42.802 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.795+0000 7f6095474000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:21:42.850 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:42.842+0000 7fb2241b5000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:21:42.865 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-10-06T11:21:42.859+0000 7f08dc43e3c0 -1 Falling back to public interface 2023-10-06T11:21:42.865 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-10-06T11:21:42.859+0000 7f32336943c0 -1 Falling back to public interface 2023-10-06T11:21:42.970 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:42.963+0000 7f6095474000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:21:43.026 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.018+0000 7fb2241b5000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:21:43.042 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:43.035+0000 7f6095474000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:21:43.093 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.086+0000 7fb2241b5000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:21:43.148 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.142+0000 7fb2241b5000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:21:43.207 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.198+0000 7fb2241b5000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:21:43.277 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:21:43.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 Processor -- start 2023-10-06T11:21:43.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- start start 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19040b2410 0x7f190406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1904069650 con 0x7f19040d07b0 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190406a490 con 0x7f190406d820 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19040ce8e0 con 0x7f19040b2410 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19040b2410 0x7f190406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19040b2410 0x7f190406dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40286/0 (socket says 172.21.15.7:40286) 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 -- 172.21.15.7:0/2650291439 learned_addr learned my addr 172.21.15.7:0/2650291439 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:43.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 -- 172.21.15.7:0/2650291439 >> v1:172.21.15.50:6789/0 conn(0x7f190406d820 legacy=0x7f19040b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 -- 172.21.15.7:0/2650291439 >> v1:172.21.15.7:6789/0 conn(0x7f19040d07b0 legacy=0x7f1904162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 -- 172.21.15.7:0/2650291439 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904069880 con 0x7f19040b2410 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f1908c1a700 1 --2- 172.21.15.7:0/2650291439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19040b2410 0x7f190406dbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18f40099b0 tx=0x7f18f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb47ae1350d02006 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f18fbfff700 1 -- 172.21.15.7:0/2650291439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400b860 con 0x7f19040b2410 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f18fbfff700 1 -- 172.21.15.7:0/2650291439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400b9c0 con 0x7f19040b2410 2023-10-06T11:21:43.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f18fbfff700 1 -- 172.21.15.7:0/2650291439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400bd60 con 0x7f19040b2410 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- 172.21.15.7:0/2650291439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19040b2410 msgr2=0x7f190406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 --2- 172.21.15.7:0/2650291439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19040b2410 0x7f190406dbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f18f40099b0 tx=0x7f18f4009250 comp rx=0 tx=0).stop 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- 172.21.15.7:0/2650291439 shutdown_connections 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 --2- 172.21.15.7:0/2650291439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19040b2410 0x7f190406dbf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- 172.21.15.7:0/2650291439 >> 172.21.15.7:0/2650291439 conn(0x7f19040689f0 msgr2=0x7f19040cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- 172.21.15.7:0/2650291439 shutdown_connections 2023-10-06T11:21:43.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.382+0000 7f190a41d700 1 -- 172.21.15.7:0/2650291439 wait complete. 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 Processor -- start 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- start start 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19040b2410 0x7f190416a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f19040d07b0 0x7f190415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040c8180 con 0x7f190406d820 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904146d30 con 0x7f19040b2410 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190416a740 con 0x7f19040d07b0 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:43.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40290/0 (socket says 172.21.15.7:40290) 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 -- 172.21.15.7:0/2405783057 learned_addr learned my addr 172.21.15.7:0/2405783057 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f1908c1a700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19040b2410 0x7f190416a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f1909c1c700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f19040d07b0 0x7f190415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 -- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f19040d07b0 msgr2=0x7f190415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f19040d07b0 0x7f190415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 -- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19040b2410 msgr2=0x7f190416a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19040b2410 0x7f190416a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 -- 172.21.15.7:0/2405783057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40094f0 con 0x7f190406d820 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190941b700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00cb40 tx=0x7f18fc0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00a7a0 con 0x7f190406d820 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00a920 con 0x7f190406d820 2023-10-06T11:21:43.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19040c9e50 con 0x7f190406d820 2023-10-06T11:21:43.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00ace0 con 0x7f190406d820 2023-10-06T11:21:43.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190415c0b0 con 0x7f190406d820 2023-10-06T11:21:43.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f18fc003030 con 0x7f190406d820 2023-10-06T11:21:43.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.386+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f18fc012c10 con 0x7f190406d820 2023-10-06T11:21:43.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.390+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1904068df0 con 0x7f190406d820 2023-10-06T11:21:43.400 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.394+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f18fc012e20 con 0x7f190406d820 2023-10-06T11:21:43.462 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.454+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19040cb6e0 con 0x7f190406d820 2023-10-06T11:21:43.463 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f18f97fa700 1 -- 172.21.15.7:0/2405783057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f18fc012630 con 0x7f190406d820 2023-10-06T11:21:43.464 INFO:teuthology.misc.health.smithi007.stdout: 2023-10-06T11:21:43.464 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:21:42.194948+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 msgr2=0x7f19041467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00cb40 tx=0x7f18fc0017b0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 shutdown_connections 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f19040d07b0 0x7f190415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19040b2410 0x7f190416a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.465 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 --2- 172.21.15.7:0/2405783057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190406d820 0x7f19041467f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:43.466 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 >> 172.21.15.7:0/2405783057 conn(0x7f19040689f0 msgr2=0x7f1904158660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:43.466 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 shutdown_connections 2023-10-06T11:21:43.466 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:43.458+0000 7f190a41d700 1 -- 172.21.15.7:0/2405783057 wait complete. 2023-10-06T11:21:43.477 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T11:21:43.487 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.482+0000 7fb2241b5000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:21:43.546 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.538+0000 7fb2241b5000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:21:43.608 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.602+0000 7fb2241b5000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:21:43.773 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:43.767+0000 7f6095474000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:21:43.774 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.766+0000 7fb2241b5000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:21:43.846 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:43.838+0000 7fb2241b5000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:21:44.579 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:21:44.574+0000 7fb2241b5000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:21:44.822 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:44.815+0000 7f607e56c700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T11:21:44.822 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:21:44.815+0000 7f607e56c700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T11:21:46.157 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-06T11:21:46.150+0000 7fd79a2ab3c0 -1 osd.3 0 log_to_monitors true 2023-10-06T11:21:46.235 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-06T11:21:46.230+0000 7f5eb5ecc3c0 -1 osd.1 0 log_to_monitors true 2023-10-06T11:21:46.267 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-10-06T11:21:46.263+0000 7f32336943c0 -1 osd.7 0 log_to_monitors true 2023-10-06T11:21:46.349 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-06T11:21:46.342+0000 7fd86b63b3c0 -1 osd.0 0 log_to_monitors true 2023-10-06T11:21:46.399 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-10-06T11:21:46.395+0000 7fba05c613c0 -1 osd.4 0 log_to_monitors true 2023-10-06T11:21:46.433 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-10-06T11:21:46.427+0000 7f04d3a5a3c0 -1 osd.6 0 log_to_monitors true 2023-10-06T11:21:46.456 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-06T11:21:46.450+0000 7ff11cc3f3c0 -1 osd.2 0 log_to_monitors true 2023-10-06T11:21:46.472 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-10-06T11:21:46.467+0000 7f08dc43e3c0 -1 osd.5 0 log_to_monitors true 2023-10-06T11:21:49.139 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-06T11:21:49.134+0000 7fd797270700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T11:21:49.144 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-06T11:21:49.138+0000 7fd78e7ef700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.147 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-06T11:21:49.142+0000 7ff1183df700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T11:21:49.155 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-06T11:21:49.150+0000 7ff111183700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.162 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-10-06T11:21:49.155+0000 7f3230659700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T11:21:49.170 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-10-06T11:21:49.163+0000 7f3227bd8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.186 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-10-06T11:21:49.183+0000 7f08d9403700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T11:21:49.244 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-10-06T11:21:49.187+0000 7fba02c26700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T11:21:49.245 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-06T11:21:49.182+0000 7fd866ddb700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T11:21:49.245 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-06T11:21:49.186+0000 7fd85fb7f700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.245 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-10-06T11:21:49.191+0000 7f08d0982700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.245 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-06T11:21:49.198+0000 7f5eb166c700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T11:21:49.246 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-06T11:21:49.206+0000 7f5eaa410700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.246 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-10-06T11:21:49.191+0000 7fb9fa1a5700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:49.246 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-10-06T11:21:49.199+0000 7f04d0a1f700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T11:21:49.246 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-10-06T11:21:49.207+0000 7f04c7f9e700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:21:50.284 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:21:50.384 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 Processor -- start 2023-10-06T11:21:50.384 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 -- start start 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faa140b20d0 0x7faa140cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 0x7faa1406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140caa30 con 0x7faa1406db70 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa140cbad0 con 0x7faa140b20d0 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19b89700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1406e010 con 0x7faa140cceb0 2023-10-06T11:21:50.385 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19388700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa1406db70 0x7faa1415c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60424/0 (socket says 172.21.15.7:60424) 2023-10-06T11:21:50.386 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faa140b20d0 0x7faa140cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.386 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa19388700 1 -- 172.21.15.7:0/1182041780 learned_addr learned my addr 172.21.15.7:0/1182041780 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa13fff700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 0x7faa1406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 -- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 msgr2=0x7faa1406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 0x7faa1406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 -- 172.21.15.7:0/1182041780 >> v1:172.21.15.7:6789/0 conn(0x7faa1406db70 legacy=0x7faa1415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa137fe700 1 -- 172.21.15.7:0/1182041780 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725771086 0 0) 0x7faa140caa30 con 0x7faa1406db70 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 -- 172.21.15.7:0/1182041780 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140cad10 con 0x7faa140b20d0 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa13fff700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 0x7faa1406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa18b87700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140b20d0 0x7faa140cd890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa040098e0 tx=0x7faa040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9c4a2feb26ce7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.387 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa137fe700 1 -- 172.21.15.7:0/1182041780 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04009e80 con 0x7faa140b20d0 2023-10-06T11:21:50.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa137fe700 1 -- 172.21.15.7:0/1182041780 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0400b860 con 0x7faa140b20d0 2023-10-06T11:21:50.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.379+0000 7faa137fe700 1 -- 172.21.15.7:0/1182041780 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0400bbe0 con 0x7faa140b20d0 2023-10-06T11:21:50.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140b20d0 msgr2=0x7faa140cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.388 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140b20d0 0x7faa140cd890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa040098e0 tx=0x7faa040091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- 172.21.15.7:0/1182041780 shutdown_connections 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140cceb0 0x7faa1406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 --2- 172.21.15.7:0/1182041780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140b20d0 0x7faa140cd890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- 172.21.15.7:0/1182041780 >> 172.21.15.7:0/1182041780 conn(0x7faa140d0790 msgr2=0x7faa140cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- 172.21.15.7:0/1182041780 shutdown_connections 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- 172.21.15.7:0/1182041780 wait complete. 2023-10-06T11:21:50.389 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 Processor -- start 2023-10-06T11:21:50.390 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.383+0000 7faa19b89700 1 -- start start 2023-10-06T11:21:50.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1406db70 0x7faa14157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140b20d0 0x7faa1415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.391 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14169ba0 con 0x7faa1406db70 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1415b060 con 0x7faa140cceb0 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa14169870 con 0x7faa140b20d0 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa18b87700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1406db70 0x7faa14157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa13fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140b20d0 0x7faa1415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:44074/0 (socket says 172.21.15.7:44074) 2023-10-06T11:21:50.392 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 -- 172.21.15.7:0/578285707 learned_addr learned my addr 172.21.15.7:0/578285707 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:50.393 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 -- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140b20d0 msgr2=0x7faa1415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.393 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140b20d0 0x7faa1415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.393 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 -- 172.21.15.7:0/578285707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1406db70 msgr2=0x7faa14157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.393 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1406db70 0x7faa14157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.393 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 -- 172.21.15.7:0/578285707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa04009420 con 0x7faa140cceb0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19388700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cc10 tx=0x7faa0c001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00a850 con 0x7faa140cceb0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00a9d0 con 0x7faa140cceb0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1415c250 con 0x7faa140cceb0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ad90 con 0x7faa140cceb0 2023-10-06T11:21:50.394 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.387+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1415c810 con 0x7faa140cceb0 2023-10-06T11:21:50.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa14169d50 con 0x7faa140cceb0 2023-10-06T11:21:50.395 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7faa0c007300 con 0x7faa140cceb0 2023-10-06T11:21:50.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa10ff9700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 0x7faa00062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7faa0c07c560 con 0x7faa140cceb0 2023-10-06T11:21:50.396 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa18b87700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 0x7faa00062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.397 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.391+0000 7faa18b87700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 0x7faa00062970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa040091e0 tx=0x7faa04017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.411 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.403+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faa14169d50 con 0x7faa140cceb0 2023-10-06T11:21:50.570 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.563+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa14169d10 con 0x7faa140cceb0 2023-10-06T11:21:50.571 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.563+0000 7faa10ff9700 1 -- 172.21.15.7:0/578285707 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10300 (secure 0 0 0) 0x7faa0c04d050 con 0x7faa140cceb0 2023-10-06T11:21:50.571 INFO:teuthology.misc.health.smithi007.stdout: 2023-10-06T11:21:50.571 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":13,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:21:50.141043+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:21:50.575 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 msgr2=0x7faa00062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.575 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 0x7faa00062970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa040091e0 tx=0x7faa04017000 comp rx=0 tx=0).stop 2023-10-06T11:21:50.575 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 msgr2=0x7faa1415bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.575 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cc10 tx=0x7faa0c001970 comp rx=0 tx=0).stop 2023-10-06T11:21:50.576 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 shutdown_connections 2023-10-06T11:21:50.576 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7faa000604c0 0x7faa00062970 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.576 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa140cceb0 0x7faa1415bc20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.576 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7faa140b20d0 0x7faa1415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.577 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 --2- 172.21.15.7:0/578285707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1406db70 0x7faa14157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.577 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 >> 172.21.15.7:0/578285707 conn(0x7faa140d0790 msgr2=0x7faa140cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:50.577 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 shutdown_connections 2023-10-06T11:21:50.577 INFO:teuthology.misc.health.smithi007.stderr:2023-10-06T11:21:50.567+0000 7faa19b89700 1 -- 172.21.15.7:0/578285707 wait complete. 2023-10-06T11:21:50.587 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T11:21:50.587 INFO:tasks.ceph:Creating RBD pool 2023-10-06T11:21:50.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T11:21:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 Processor -- start 2023-10-06T11:21:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- start start 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 0x7f00780c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f007815d300 0x7f00780c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780ca6d0 con 0x7f0078158c00 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780caee0 con 0x7f007815d300 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007815d720 con 0x7f0078158fd0 2023-10-06T11:21:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f007815d300 0x7f00780c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f00777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0078158c00 0x7f00780c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60428/0 (socket says 172.21.15.7:60428) 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f00777fe700 1 -- 172.21.15.7:0/2992113577 learned_addr learned my addr 172.21.15.7:0/2992113577 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0076ffd700 1 --2- 172.21.15.7:0/2992113577 >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 0x7f00780c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 -- 172.21.15.7:0/2992113577 >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 msgr2=0x7f00780c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 --2- 172.21.15.7:0/2992113577 >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 0x7f00780c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 -- 172.21.15.7:0/2992113577 >> v1:172.21.15.7:6789/0 conn(0x7f0078158c00 legacy=0x7f00780c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 -- 172.21.15.7:0/2992113577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780ca920 con 0x7f007815d300 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0076ffd700 1 --2- 172.21.15.7:0/2992113577 >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 0x7f00780c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:21:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f0077fff700 1 --2- 172.21.15.7:0/2992113577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007815d300 0x7f00780c8800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00780b2160 tx=0x7f0070009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c560d5a79b6f371 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f00767fc700 1 -- 172.21.15.7:0/2992113577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000e070 con 0x7f007815d300 2023-10-06T11:21:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f00767fc700 1 -- 172.21.15.7:0/2992113577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007000bc30 con 0x7f007815d300 2023-10-06T11:21:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f00767fc700 1 -- 172.21.15.7:0/2992113577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070012430 con 0x7f007815d300 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- 172.21.15.7:0/2992113577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007815d300 msgr2=0x7f00780c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 --2- 172.21.15.7:0/2992113577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007815d300 0x7f00780c8800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00780b2160 tx=0x7f0070009780 comp rx=0 tx=0).stop 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- 172.21.15.7:0/2992113577 shutdown_connections 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 --2- 172.21.15.7:0/2992113577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007815d300 0x7f00780c8800 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 --2- 172.21.15.7:0/2992113577 >> v2:172.21.15.50:3300/0 conn(0x7f0078158fd0 0x7f00780c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- 172.21.15.7:0/2992113577 >> 172.21.15.7:0/2992113577 conn(0x7f007806a9d0 msgr2=0x7f0078067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- 172.21.15.7:0/2992113577 shutdown_connections 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.679+0000 7f007c98b700 1 -- 172.21.15.7:0/2992113577 wait complete. 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 Processor -- start 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- start start 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078158c00 0x7f00780c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f007815d300 0x7f00780b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c4120 con 0x7f0078158fd0 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c6090 con 0x7f0078158c00 2023-10-06T11:21:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c56c0 con 0x7f007815d300 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f00777fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078158c00 0x7f00780c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40356/0 (socket says 172.21.15.7:40356) 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0077fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f007815d300 0x7f00780b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 -- 172.21.15.7:0/796280654 learned_addr learned my addr 172.21.15.7:0/796280654 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 -- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f007815d300 msgr2=0x7f00780b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f007815d300 0x7f00780b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 -- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078158c00 msgr2=0x7f00780c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078158c00 0x7f00780c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 -- 172.21.15.7:0/796280654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070009a20 con 0x7f0078158fd0 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f0076ffd700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c860 tx=0x7f006c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c011600 con 0x7f0078158fd0 2023-10-06T11:21:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c011760 con 0x7f0078158fd0 2023-10-06T11:21:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007814c230 con 0x7f0078158fd0 2023-10-06T11:21:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780c5070 con 0x7f0078158fd0 2023-10-06T11:21:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.683+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c011b20 con 0x7f0078158fd0 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.687+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f006c01a450 con 0x7f0078158fd0 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.687+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007806d170 con 0x7f0078158fd0 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.687+0000 7f005f7fe700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 0x7f00640629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.687+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f006c07c4b0 con 0x7f0078158fd0 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.687+0000 7f00777fe700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 0x7f00640629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.691+0000 7f00777fe700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 0x7f00640629a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.695+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f006c04d030 con 0x7f0078158fd0 2023-10-06T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:50.855+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f00780c5800 con 0x7f0078158fd0 2023-10-06T11:21:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.151+0000 7f005f7fe700 1 -- 172.21.15.7:0/796280654 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f006c050650 con 0x7f0078158fd0 2023-10-06T11:21:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 msgr2=0x7f00640629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 0x7f00640629a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068009040 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 msgr2=0x7f00780c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c860 tx=0x7f006c008040 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 shutdown_connections 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f00640604f0 0x7f00640629a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f007815d300 0x7f00780b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078158fd0 0x7f00780c5b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 --2- 172.21.15.7:0/796280654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078158c00 0x7f00780c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 >> 172.21.15.7:0/796280654 conn(0x7f007806a9d0 msgr2=0x7f007806b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:51.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 shutdown_connections 2023-10-06T11:21:51.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.155+0000 7f007c98b700 1 -- 172.21.15.7:0/796280654 wait complete. 2023-10-06T11:21:51.163 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2023-10-06T11:21:51.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T11:21:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.283+0000 7ffbeb59e700 1 Processor -- start 2023-10-06T11:21:51.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.283+0000 7ffbeb59e700 1 -- start start 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffbec0ae690 0x7ffbec1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0c8520 0x7ffbec0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec0aeb90 con 0x7ffbec159120 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec0b1560 con 0x7ffbec0ae690 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0aee70 con 0x7ffbec0c8520 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbea59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffbec159120 0x7ffbec0adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60440/0 (socket says 172.21.15.7:60440) 2023-10-06T11:21:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbea59c700 1 -- 172.21.15.7:0/1651464287 learned_addr learned my addr 172.21.15.7:0/1651464287 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 --2- 172.21.15.7:0/1651464287 >> v2:172.21.15.50:3300/0 conn(0x7ffbec0ae690 0x7ffbec1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbead9d700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0c8520 0x7ffbec0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe959a700 1 -- 172.21.15.7:0/1651464287 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587833421 0 0) 0x7ffbec0aeb90 con 0x7ffbec159120 2023-10-06T11:21:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe959a700 1 -- 172.21.15.7:0/1651464287 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd8003570 con 0x7ffbec159120 2023-10-06T11:21:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 -- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0c8520 msgr2=0x7ffbec0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0c8520 0x7ffbec0c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 -- 172.21.15.7:0/1651464287 >> v1:172.21.15.7:6789/0 conn(0x7ffbec159120 legacy=0x7ffbec0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 -- 172.21.15.7:0/1651464287 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec0af150 con 0x7ffbec0ae690 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe9d9b700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0ae690 0x7ffbec1594f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400aa90 tx=0x7ffbd400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31be9ca46b422b60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe959a700 1 -- 172.21.15.7:0/1651464287 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400c970 con 0x7ffbec0ae690 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe959a700 1 -- 172.21.15.7:0/1651464287 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd400cad0 con 0x7ffbec0ae690 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbe959a700 1 -- 172.21.15.7:0/1651464287 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400ce50 con 0x7ffbec0ae690 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0ae690 msgr2=0x7ffbec1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0ae690 0x7ffbec1594f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400aa90 tx=0x7ffbd400a330 comp rx=0 tx=0).stop 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- 172.21.15.7:0/1651464287 shutdown_connections 2023-10-06T11:21:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0c8520 0x7ffbec0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/1651464287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0ae690 0x7ffbec1594f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- 172.21.15.7:0/1651464287 >> 172.21.15.7:0/1651464287 conn(0x7ffbec068880 msgr2=0x7ffbec0cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- 172.21.15.7:0/1651464287 shutdown_connections 2023-10-06T11:21:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.287+0000 7ffbeb59e700 1 -- 172.21.15.7:0/1651464287 wait complete. 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 Processor -- start 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- start start 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0ae690 0x7ffbec148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 0x7ffbec15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec149260 con 0x7ffbec159120 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec15b570 con 0x7ffbec0c8520 2023-10-06T11:21:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec15b850 con 0x7ffbec0ae690 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbe9d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 0x7ffbec15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40366/0 (socket says 172.21.15.7:40366) 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 -- 172.21.15.7:0/3971463146 learned_addr learned my addr 172.21.15.7:0/3971463146 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbea59c700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0ae690 0x7ffbec148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 -- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0ae690 msgr2=0x7ffbec148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0ae690 0x7ffbec148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 -- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 msgr2=0x7ffbec15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 0x7ffbec15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 -- 172.21.15.7:0/3971463146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd400a5d0 con 0x7ffbec159120 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbe9d9b700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 0x7ffbec15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbead9d700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009e90 tx=0x7ffbdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00cb70 con 0x7ffbec159120 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00f040 con 0x7ffbec159120 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec15bba0 con 0x7ffbec159120 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc014480 con 0x7ffbec159120 2023-10-06T11:21:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.291+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec15c160 con 0x7ffbec159120 2023-10-06T11:21:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec06c410 con 0x7ffbec159120 2023-10-06T11:21:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7ffbdc014830 con 0x7ffbec159120 2023-10-06T11:21:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbe27fc700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 0x7ffbcc062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4809+0+0 (secure 0 0 0) 0x7ffbdc07cd50 con 0x7ffbec159120 2023-10-06T11:21:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbea59c700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 0x7ffbcc062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:51.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.295+0000 7ffbea59c700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 0x7ffbcc062970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4007fd0 tx=0x7ffbe4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:51.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.303+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffbdc04d850 con 0x7ffbec159120 2023-10-06T11:21:51.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:51.471+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffbec0af430 con 0x7ffbec159120 2023-10-06T11:21:52.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.155+0000 7ffbe27fc700 1 -- 172.21.15.7:0/3971463146 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7ffbdc0180c0 con 0x7ffbec159120 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 msgr2=0x7ffbcc062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 0x7ffbcc062970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4007fd0 tx=0x7ffbe4009040 comp rx=0 tx=0).stop 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 msgr2=0x7ffbec15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009e90 tx=0x7ffbdc00b040 comp rx=0 tx=0).stop 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 shutdown_connections 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffbcc0604c0 0x7ffbcc062970 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4007fd0 tx=0x7ffbe4009040 comp rx=0 tx=0).stop 2023-10-06T11:21:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbec159120 0x7ffbec15e9f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbec0c8520 0x7ffbec15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 --2- 172.21.15.7:0/3971463146 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffbec0ae690 0x7ffbec148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 >> 172.21.15.7:0/3971463146 conn(0x7ffbec068880 msgr2=0x7ffbec069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 shutdown_connections 2023-10-06T11:21:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.163+0000 7ffbeb59e700 1 -- 172.21.15.7:0/3971463146 wait complete. 2023-10-06T11:21:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T11:21:52.182 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T11:21:52.182 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:21:52.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:21:52.185 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:21:52.185 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:21:52.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:21:52.187 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:21:52.187 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:21:52.187 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:21:52.189 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:21:52.190 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:21:52.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:21:52.192 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:21:52.192 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T11:21:52.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:21:52.253 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-06T11:21:52.256 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-06T11:21:52.271 INFO:tasks.ceph.mds.b.smithi050.stdout:starting mds.b at 2023-10-06T11:21:52.281 INFO:tasks.ceph.mds.d.smithi050.stdout:starting mds.d at 2023-10-06T11:21:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.287+0000 7fd862067700 1 Processor -- start 2023-10-06T11:21:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.287+0000 7fd862067700 1 -- start start 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.287+0000 7fd862067700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.287+0000 7fd862067700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c1597e0 con 0x7fd85c153a00 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.287+0000 7fd862067700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c15a080 con 0x7fd85c153630 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c156ff0 con 0x7fd85c0cc090 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd85c153a00 0x7fd85c1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55098/0 (socket says 172.21.15.7:55098) 2023-10-06T11:21:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85affd700 1 -- 172.21.15.7:0/2698763288 learned_addr learned my addr 172.21.15.7:0/2698763288 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85bfff700 1 --2- 172.21.15.7:0/2698763288 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85bfff700 1 -- 172.21.15.7:0/2698763288 >> v1:172.21.15.50:6789/0 conn(0x7fd85c153630 legacy=0x7fd85c0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85bfff700 1 -- 172.21.15.7:0/2698763288 >> v1:172.21.15.7:6789/0 conn(0x7fd85c153a00 legacy=0x7fd85c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85bfff700 1 -- 172.21.15.7:0/2698763288 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c159a30 con 0x7fd85c0cc090 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85bfff700 1 --2- 172.21.15.7:0/2698763288 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c1569d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd854009e90 tx=0x7fd854009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1929107b494e51a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85a7fc700 1 -- 172.21.15.7:0/2698763288 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85400e070 con 0x7fd85c0cc090 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85a7fc700 1 -- 172.21.15.7:0/2698763288 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85400bc30 con 0x7fd85c0cc090 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd85a7fc700 1 -- 172.21.15.7:0/2698763288 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854012430 con 0x7fd85c0cc090 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- 172.21.15.7:0/2698763288 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 msgr2=0x7fd85c1569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 --2- 172.21.15.7:0/2698763288 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c1569d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd854009e90 tx=0x7fd854009780 comp rx=0 tx=0).stop 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- 172.21.15.7:0/2698763288 shutdown_connections 2023-10-06T11:21:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 --2- 172.21.15.7:0/2698763288 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c1569d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- 172.21.15.7:0/2698763288 >> 172.21.15.7:0/2698763288 conn(0x7fd85c06b8a0 msgr2=0x7fd85c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- 172.21.15.7:0/2698763288 shutdown_connections 2023-10-06T11:21:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- 172.21.15.7:0/2698763288 wait complete. 2023-10-06T11:21:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 Processor -- start 2023-10-06T11:21:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.291+0000 7fd862067700 1 -- start start 2023-10-06T11:21:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd85c153630 0x7fd85c0c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c149a40 con 0x7fd85c153a00 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c149010 con 0x7fd85c153630 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0caca0 con 0x7fd85c0cc090 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54994/0 (socket says 172.21.15.7:54994) 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 -- 172.21.15.7:0/3581320042 learned_addr learned my addr 172.21.15.7:0/3581320042 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85affd700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd85c153630 0x7fd85c0c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 -- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 msgr2=0x7fd85c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 -- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd85c153630 msgr2=0x7fd85c0c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd85c153630 0x7fd85c0c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 -- 172.21.15.7:0/3581320042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854009a20 con 0x7fd85c153a00 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd85bfff700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd854013000 tx=0x7fd854019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854012e10 con 0x7fd85c153a00 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85401d630 con 0x7fd85c153a00 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85401d9f0 con 0x7fd85c153a00 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c166a50 con 0x7fd85c153a00 2023-10-06T11:21:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0c9030 con 0x7fd85c153a00 2023-10-06T11:21:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c166be0 con 0x7fd85c153a00 2023-10-06T11:21:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.295+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fd854024070 con 0x7fd85c153a00 2023-10-06T11:21:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.299+0000 7fd8437fe700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 0x7fd848062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.299+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 0x7fd848062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.299+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fd85407ec80 con 0x7fd85c153a00 2023-10-06T11:21:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.299+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 0x7fd848062970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd850003a80 tx=0x7fd850007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.303+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd85404f550 con 0x7fd85c153a00 2023-10-06T11:21:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.467+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85c04be30 con 0x7fd85c153a00 2023-10-06T11:21:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.471+0000 7fd8437fe700 1 -- 172.21.15.7:0/3581320042 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13570 (secure 0 0 0) 0x7fd85402c020 con 0x7fd85c153a00 2023-10-06T11:21:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:52.476 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:21:52.147747+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:21:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.471+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 msgr2=0x7fd848062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.471+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 0x7fd848062970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd850003a80 tx=0x7fd850007ec0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.471+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 msgr2=0x7fd85c1522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.471+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd854013000 tx=0x7fd854019000 comp rx=0 tx=0).stop 2023-10-06T11:21:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 shutdown_connections 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd8480604c0 0x7fd848062970 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c153a00 0x7fd85c1522d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd85c153630 0x7fd85c0c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 --2- 172.21.15.7:0/3581320042 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd85c0cc090 0x7fd85c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 >> 172.21.15.7:0/3581320042 conn(0x7fd85c06b8a0 msgr2=0x7fd85c157570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 shutdown_connections 2023-10-06T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.475+0000 7fd862067700 1 -- 172.21.15.7:0/3581320042 wait complete. 2023-10-06T11:21:52.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2023-10-06T11:21:52.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:21:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 Processor -- start 2023-10-06T11:21:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 -- start start 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f16000cd450 0x7f16000cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000cc3b0 con 0x7f16000ac9f0 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000cc690 con 0x7f160006d6e0 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.603+0000 7f1605f1c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1600157320 con 0x7f16000cd450 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1604f1a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16000ac9f0 0x7f160006cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55110/0 (socket says 172.21.15.7:55110) 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1604f1a700 1 -- 172.21.15.7:0/3907867496 learned_addr learned my addr 172.21.15.7:0/3907867496 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f160571b700 1 --2- 172.21.15.7:0/3907867496 >> v2:172.21.15.50:3300/0 conn(0x7f16000cd450 0x7f16000cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f15ff7fe700 1 -- 172.21.15.7:0/3907867496 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110997454 0 0) 0x7f16000cc3b0 con 0x7f16000ac9f0 2023-10-06T11:21:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f15ff7fe700 1 -- 172.21.15.7:0/3907867496 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e8003570 con 0x7f16000ac9f0 2023-10-06T11:21:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f160571b700 1 -- 172.21.15.7:0/3907867496 >> v1:172.21.15.50:6789/0 conn(0x7f160006d6e0 legacy=0x7f16000cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f160571b700 1 -- 172.21.15.7:0/3907867496 >> v1:172.21.15.7:6789/0 conn(0x7f16000ac9f0 legacy=0x7f160006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f160571b700 1 -- 172.21.15.7:0/3907867496 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000cc970 con 0x7f16000cd450 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f160571b700 1 --2- 172.21.15.7:0/3907867496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000cbdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009e60 tx=0x7f15f8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a62250ea1555d93e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f15ff7fe700 1 -- 172.21.15.7:0/3907867496 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800e070 con 0x7f16000cd450 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f15ff7fe700 1 -- 172.21.15.7:0/3907867496 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800bb60 con 0x7f16000cd450 2023-10-06T11:21:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f15ff7fe700 1 -- 172.21.15.7:0/3907867496 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8012430 con 0x7f16000cd450 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 -- 172.21.15.7:0/3907867496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 msgr2=0x7f16000cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 --2- 172.21.15.7:0/3907867496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000cbdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009e60 tx=0x7f15f8009780 comp rx=0 tx=0).stop 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 -- 172.21.15.7:0/3907867496 shutdown_connections 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 --2- 172.21.15.7:0/3907867496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000cbdb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 -- 172.21.15.7:0/3907867496 >> 172.21.15.7:0/3907867496 conn(0x7f16000ce570 msgr2=0x7f16000ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 -- 172.21.15.7:0/3907867496 shutdown_connections 2023-10-06T11:21:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.607+0000 7f1605f1c700 1 -- 172.21.15.7:0/3907867496 wait complete. 2023-10-06T11:21:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 Processor -- start 2023-10-06T11:21:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- start start 2023-10-06T11:21:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f16000ac9f0 0x7f16000aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600068fe0 con 0x7f160006d6e0 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600069280 con 0x7f16000cd450 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1600069560 con 0x7f16000ac9f0 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54996/0 (socket says 172.21.15.7:54996) 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 -- 172.21.15.7:0/2932893903 learned_addr learned my addr 172.21.15.7:0/2932893903 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f160571b700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f15fffff700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f16000ac9f0 0x7f16000aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 -- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f16000ac9f0 msgr2=0x7f16000aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f16000ac9f0 0x7f16000aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 -- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 msgr2=0x7f16000aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 -- 172.21.15.7:0/2932893903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009a20 con 0x7f160006d6e0 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1604f1a700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c7c0 tx=0x7f15f4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005cd0 con 0x7f160006d6e0 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4005e30 con 0x7f160006d6e0 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1600069840 con 0x7f160006d6e0 2023-10-06T11:21:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4007320 con 0x7f160006d6e0 2023-10-06T11:21:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.611+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160014df40 con 0x7f160006d6e0 2023-10-06T11:21:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.615+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160006eb50 con 0x7f160006d6e0 2023-10-06T11:21:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.615+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f15f401b3b0 con 0x7f160006d6e0 2023-10-06T11:21:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.615+0000 7f15fcff9700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 0x7f15ec062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.615+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f15f407bbe0 con 0x7f160006d6e0 2023-10-06T11:21:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.615+0000 7f15fffff700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 0x7f15ec062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.619+0000 7f15fffff700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 0x7f15ec062970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16000af5a0 tx=0x7f15f0000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.623+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f15f404c410 con 0x7f160006d6e0 2023-10-06T11:21:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.779+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16000ccc50 con 0x7f160006d6e0 2023-10-06T11:21:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f15fcff9700 1 -- 172.21.15.7:0/2932893903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f15f4017020 con 0x7f160006d6e0 2023-10-06T11:21:52.788 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 msgr2=0x7f15ec062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 0x7f15ec062970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16000af5a0 tx=0x7f15f0000bc0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 msgr2=0x7f160014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c7c0 tx=0x7f15f4007e00 comp rx=0 tx=0).stop 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 shutdown_connections 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f15ec0604c0 0x7f15ec062970 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16000cd450 0x7f16000aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f16000ac9f0 0x7f16000aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 --2- 172.21.15.7:0/2932893903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160006d6e0 0x7f160014da00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 >> 172.21.15.7:0/2932893903 conn(0x7f16000ce570 msgr2=0x7f160014e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 shutdown_connections 2023-10-06T11:21:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.783+0000 7f1605f1c700 1 -- 172.21.15.7:0/2932893903 wait complete. 2023-10-06T11:21:52.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:21:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.911+0000 7f751d1f4700 1 Processor -- start 2023-10-06T11:21:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.911+0000 7f751d1f4700 1 -- start start 2023-10-06T11:21:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7518159030 0x7f7518067e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75180c7980 con 0x7f7518158c60 2023-10-06T11:21:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7518068780 con 0x7f7518068370 2023-10-06T11:21:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751806cd70 con 0x7f7518159030 2023-10-06T11:21:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f7517fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7518158c60 0x7f751806c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55126/0 (socket says 172.21.15.7:55126) 2023-10-06T11:21:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f7517fff700 1 -- 172.21.15.7:0/1429692581 learned_addr learned my addr 172.21.15.7:0/1429692581 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f750f7fe700 1 --2- 172.21.15.7:0/1429692581 >> v2:172.21.15.50:3300/0 conn(0x7f7518159030 0x7f7518067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581774945 0 0) 0x7f75180c7980 con 0x7f7518158c60 2023-10-06T11:21:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f8003570 con 0x7f7518158c60 2023-10-06T11:21:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3021508205 0 0) 0x7f74f8003570 con 0x7f7518158c60 2023-10-06T11:21:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f750f7fe700 1 -- 172.21.15.7:0/1429692581 >> v1:172.21.15.50:6789/0 conn(0x7f7518068370 legacy=0x7f75180c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f750f7fe700 1 -- 172.21.15.7:0/1429692581 >> v1:172.21.15.7:6789/0 conn(0x7f7518158c60 legacy=0x7f751806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f750f7fe700 1 -- 172.21.15.7:0/1429692581 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751806d050 con 0x7f7518159030 2023-10-06T11:21:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f750f7fe700 1 --2- 172.21.15.7:0/1429692581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518159030 0x7f7518067e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75000098e0 tx=0x7f75000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04bd26782992e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508001470 con 0x7f7518159030 2023-10-06T11:21:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7500011420 con 0x7f7518159030 2023-10-06T11:21:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f75177fe700 1 -- 172.21.15.7:0/1429692581 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75000117c0 con 0x7f7518159030 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- 172.21.15.7:0/1429692581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518159030 msgr2=0x7f7518067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 --2- 172.21.15.7:0/1429692581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518159030 0x7f7518067e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75000098e0 tx=0x7f75000091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- 172.21.15.7:0/1429692581 shutdown_connections 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 --2- 172.21.15.7:0/1429692581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518159030 0x7f7518067e30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- 172.21.15.7:0/1429692581 >> 172.21.15.7:0/1429692581 conn(0x7f751806f2a0 msgr2=0x7f7518068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- 172.21.15.7:0/1429692581 shutdown_connections 2023-10-06T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.915+0000 7f751d1f4700 1 -- 172.21.15.7:0/1429692581 wait complete. 2023-10-06T11:21:52.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 Processor -- start 2023-10-06T11:21:52.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 -- start start 2023-10-06T11:21:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7518068370 0x7f7518153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518158c60 0x7f7518152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518154580 con 0x7f75180ae7a0 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518153970 con 0x7f7518158c60 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751d1f4700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7518153c50 con 0x7f7518068370 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55000/0 (socket says 172.21.15.7:55000) 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 -- 172.21.15.7:0/3855040234 learned_addr learned my addr 172.21.15.7:0/3855040234 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f750f7fe700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518158c60 0x7f7518152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f7517fff700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7518068370 0x7f7518153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 -- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7518068370 msgr2=0x7f7518153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7518068370 0x7f7518153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 -- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518158c60 msgr2=0x7f7518152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518158c60 0x7f7518152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.919+0000 7f751c9f3700 1 -- 172.21.15.7:0/3855040234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009420 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f751c9f3700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f751000cb90 tx=0x7f75100017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75100020b0 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751000a7e0 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518153f30 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751000abc0 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7518153680 con 0x7f75180ae7a0 2023-10-06T11:21:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180caf80 con 0x7f75180ae7a0 2023-10-06T11:21:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.923+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f7510003030 con 0x7f75180ae7a0 2023-10-06T11:21:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.927+0000 7f7514ff9700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 0x7f74fc062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.927+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f751007c5a0 con 0x7f75180ae7a0 2023-10-06T11:21:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.927+0000 7f7517fff700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 0x7f74fc062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.927+0000 7f7517fff700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 0x7f74fc062970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f75080011e0 tx=0x7f7508000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:52.931+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f75180caf80 con 0x7f75180ae7a0 2023-10-06T11:21:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.087+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75180caf80 con 0x7f75180ae7a0 2023-10-06T11:21:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f7514ff9700 1 -- 172.21.15.7:0/3855040234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75180caf80 con 0x7f75180ae7a0 2023-10-06T11:21:53.095 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:21:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 msgr2=0x7f74fc062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 0x7f74fc062970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f75080011e0 tx=0x7f7508000bc0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 msgr2=0x7f75180c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f751000cb90 tx=0x7f75100017c0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 shutdown_connections 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74fc0604c0 0x7f74fc062970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75180ae7a0 0x7f75180c2750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7518158c60 0x7f7518152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 --2- 172.21.15.7:0/3855040234 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7518068370 0x7f7518153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 >> 172.21.15.7:0/3855040234 conn(0x7f751806f2a0 msgr2=0x7f75180c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 shutdown_connections 2023-10-06T11:21:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.091+0000 7f751d1f4700 1 -- 172.21.15.7:0/3855040234 wait complete. 2023-10-06T11:21:53.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:21:53.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.211+0000 7f2e32662700 1 Processor -- start 2023-10-06T11:21:53.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.211+0000 7f2e32662700 1 -- start start 2023-10-06T11:21:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e2c158b40 0x7f2e2c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e2c0ccc60 0x7f2e2c069720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c06e310 con 0x7f2e2c0cb1e0 2023-10-06T11:21:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c06e7a0 con 0x7f2e2c158b40 2023-10-06T11:21:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c06d720 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e30e5f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e2c0cb1e0 0x7f2e2c0cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55140/0 (socket says 172.21.15.7:55140) 2023-10-06T11:21:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e30e5f700 1 -- 172.21.15.7:0/1770917428 learned_addr learned my addr 172.21.15.7:0/1770917428 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 --2- 172.21.15.7:0/1770917428 >> v2:172.21.15.7:3300/0 conn(0x7f2e2c158b40 0x7f2e2c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31e61700 1 --2- 172.21.15.7:0/1770917428 >> v2:172.21.15.50:3300/0 conn(0x7f2e2c0ccc60 0x7f2e2c069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e23fff700 1 -- 172.21.15.7:0/1770917428 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265802582 0 0) 0x7f2e2c06e310 con 0x7f2e2c0cb1e0 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e23fff700 1 -- 172.21.15.7:0/1770917428 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e18003570 con 0x7f2e2c0cb1e0 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 -- 172.21.15.7:0/1770917428 >> v2:172.21.15.50:3300/0 conn(0x7f2e2c0ccc60 msgr2=0x7f2e2c069720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 --2- 172.21.15.7:0/1770917428 >> v2:172.21.15.50:3300/0 conn(0x7f2e2c0ccc60 0x7f2e2c069720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 -- 172.21.15.7:0/1770917428 >> v1:172.21.15.7:6789/0 conn(0x7f2e2c0cb1e0 legacy=0x7f2e2c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 -- 172.21.15.7:0/1770917428 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c06d9e0 con 0x7f2e2c158b40 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e31660700 1 --2- 172.21.15.7:0/1770917428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c158b40 0x7f2e2c0caca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c15c900 tx=0x7f2e280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3356cf02445892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e23fff700 1 -- 172.21.15.7:0/1770917428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28011630 con 0x7f2e2c158b40 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e23fff700 1 -- 172.21.15.7:0/1770917428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28011790 con 0x7f2e2c158b40 2023-10-06T11:21:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e23fff700 1 -- 172.21.15.7:0/1770917428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28011b50 con 0x7f2e2c158b40 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- 172.21.15.7:0/1770917428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c158b40 msgr2=0x7f2e2c0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 --2- 172.21.15.7:0/1770917428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c158b40 0x7f2e2c0caca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c15c900 tx=0x7f2e280091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- 172.21.15.7:0/1770917428 shutdown_connections 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 --2- 172.21.15.7:0/1770917428 >> v2:172.21.15.50:3300/0 conn(0x7f2e2c0ccc60 0x7f2e2c069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 --2- 172.21.15.7:0/1770917428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c158b40 0x7f2e2c0caca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- 172.21.15.7:0/1770917428 >> 172.21.15.7:0/1770917428 conn(0x7f2e2c067e30 msgr2=0x7f2e2c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- 172.21.15.7:0/1770917428 shutdown_connections 2023-10-06T11:21:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.215+0000 7f2e32662700 1 -- 172.21.15.7:0/1770917428 wait complete. 2023-10-06T11:21:53.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 Processor -- start 2023-10-06T11:21:53.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- start start 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e2c0cb1e0 0x7f2e2c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e2c158b40 0x7f2e2c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c14cfa0 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b01c0 con 0x7f2e2c0cb1e0 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b04a0 con 0x7f2e2c158b40 2023-10-06T11:21:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55010/0 (socket says 172.21.15.7:55010) 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 -- 172.21.15.7:0/3163139733 learned_addr learned my addr 172.21.15.7:0/3163139733 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e31660700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e2c0cb1e0 0x7f2e2c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e31e61700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e2c158b40 0x7f2e2c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 -- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e2c158b40 msgr2=0x7f2e2c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e2c158b40 0x7f2e2c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 -- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e2c0cb1e0 msgr2=0x7f2e2c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e2c0cb1e0 0x7f2e2c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 -- 172.21.15.7:0/3163139733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009420 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e30e5f700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c7c0 tx=0x7f2e1c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005cd0 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c005e30 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c0b0780 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007320 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.219+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c155ad0 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c155930 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f2e1c01b3b0 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e217fa700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 0x7f2e10062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2e1c07bb50 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e31660700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 0x7f2e10062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.223+0000 7f2e31660700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 0x7f2e10062970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c15c900 tx=0x7f2e28017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.231+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2e2c155930 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.431+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e2c155930 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.431+0000 7f2e217fa700 1 -- 172.21.15.7:0/3163139733 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3515 (secure 0 0 0) 0x7f2e2c155930 con 0x7f2e2c0ccc60 2023-10-06T11:21:53.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:53.438 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T11:21:53.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 msgr2=0x7f2e10062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 0x7f2e10062970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c15c900 tx=0x7f2e28017000 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 msgr2=0x7f2e2c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c7c0 tx=0x7f2e1c007e00 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 shutdown_connections 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e100604c0 0x7f2e10062970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e2c158b40 0x7f2e2c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e2c0ccc60 0x7f2e2c0b15c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 --2- 172.21.15.7:0/3163139733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e2c0cb1e0 0x7f2e2c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 >> 172.21.15.7:0/3163139733 conn(0x7f2e2c067e30 msgr2=0x7f2e2c14d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 shutdown_connections 2023-10-06T11:21:53.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.435+0000 7f2e32662700 1 -- 172.21.15.7:0/3163139733 wait complete. 2023-10-06T11:21:53.442 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2023-10-06T11:21:53.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:21:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 Processor -- start 2023-10-06T11:21:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 -- start start 2023-10-06T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f394c155840 0x7f394c159670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f394c06c0b0 0x7f394c0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c0c8c20 con 0x7f394c159bb0 2023-10-06T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c15a610 con 0x7f394c06c0b0 2023-10-06T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3952fbe700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c0c8610 con 0x7f394c155840 2023-10-06T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f39517bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f394c06c0b0 0x7f394c0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f39517bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f394c06c0b0 0x7f394c0cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55024/0 (socket says 172.21.15.7:55024) 2023-10-06T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f39517bb700 1 -- 172.21.15.7:0/3324546028 learned_addr learned my addr 172.21.15.7:0/3324546028 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.551+0000 7f3951fbc700 1 --2- 172.21.15.7:0/3324546028 >> v2:172.21.15.50:3300/0 conn(0x7f394c155840 0x7f394c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f39517bb700 1 -- 172.21.15.7:0/3324546028 >> v2:172.21.15.50:3300/0 conn(0x7f394c155840 msgr2=0x7f394c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f39517bb700 1 --2- 172.21.15.7:0/3324546028 >> v2:172.21.15.50:3300/0 conn(0x7f394c155840 0x7f394c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f39517bb700 1 -- 172.21.15.7:0/3324546028 >> v1:172.21.15.50:6789/0 conn(0x7f394c159bb0 legacy=0x7f394c0ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f39517bb700 1 -- 172.21.15.7:0/3324546028 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c15a050 con 0x7f394c06c0b0 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f39517bb700 1 --2- 172.21.15.7:0/3324546028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c06c0b0 0x7f394c0cc9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f394c067dc0 tx=0x7f3940009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803229ce39e6ce96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3950fba700 1 -- 172.21.15.7:0/3324546028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940009e70 con 0x7f394c06c0b0 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3950fba700 1 -- 172.21.15.7:0/3324546028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000b860 con 0x7f394c06c0b0 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3950fba700 1 -- 172.21.15.7:0/3324546028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000bc00 con 0x7f394c06c0b0 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- 172.21.15.7:0/3324546028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c06c0b0 msgr2=0x7f394c0cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- 172.21.15.7:0/3324546028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c06c0b0 0x7f394c0cc9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f394c067dc0 tx=0x7f3940009250 comp rx=0 tx=0).stop 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- 172.21.15.7:0/3324546028 shutdown_connections 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- 172.21.15.7:0/3324546028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c06c0b0 0x7f394c0cc9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- 172.21.15.7:0/3324546028 >> v2:172.21.15.50:3300/0 conn(0x7f394c155840 0x7f394c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- 172.21.15.7:0/3324546028 >> 172.21.15.7:0/3324546028 conn(0x7f394c06f380 msgr2=0x7f394c06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- 172.21.15.7:0/3324546028 shutdown_connections 2023-10-06T11:21:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- 172.21.15.7:0/3324546028 wait complete. 2023-10-06T11:21:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 Processor -- start 2023-10-06T11:21:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- start start 2023-10-06T11:21:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f394c155840 0x7f394c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 0x7f394c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0ca3a0 con 0x7f394c159bb0 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0caff0 con 0x7f394c155840 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.555+0000 7f3952fbe700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0c9f10 con 0x7f394c0ca650 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55040/0 (socket says 172.21.15.7:55040) 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 -- 172.21.15.7:0/2138145578 learned_addr learned my addr 172.21.15.7:0/2138145578 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3951fbc700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f394c155840 0x7f394c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39527bd700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 0x7f394c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 -- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 msgr2=0x7f394c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 0x7f394c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 -- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f394c155840 msgr2=0x7f394c1504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f394c155840 0x7f394c1504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 -- 172.21.15.7:0/2138145578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400094f0 con 0x7f394c159bb0 2023-10-06T11:21:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39527bd700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 0x7f394c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f39517bb700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f394c04c480 tx=0x7f3940017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000b9d0 con 0x7f394c159bb0 2023-10-06T11:21:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000bb30 con 0x7f394c159bb0 2023-10-06T11:21:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c0cd4a0 con 0x7f394c159bb0 2023-10-06T11:21:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940025650 con 0x7f394c159bb0 2023-10-06T11:21:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c0c95e0 con 0x7f394c159bb0 2023-10-06T11:21:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c0ca0a0 con 0x7f394c159bb0 2023-10-06T11:21:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.559+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f3940023070 con 0x7f394c159bb0 2023-10-06T11:21:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.563+0000 7f3949ffb700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 0x7f3938062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.563+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f394007de50 con 0x7f394c159bb0 2023-10-06T11:21:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.563+0000 7f3951fbc700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 0x7f3938062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.563+0000 7f3951fbc700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 0x7f3938062970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f393c009db0 tx=0x7f393c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.567+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f394004e790 con 0x7f394c159bb0 2023-10-06T11:21:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.727+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394c15a330 con 0x7f394c159bb0 2023-10-06T11:21:53.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.727+0000 7f3949ffb700 1 -- 172.21.15.7:0/2138145578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13565 (secure 0 0 0) 0x7f3940015090 con 0x7f394c159bb0 2023-10-06T11:21:53.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:53.735 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:21:53.162715+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:21:53.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 msgr2=0x7f3938062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 0x7f3938062970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f393c009db0 tx=0x7f393c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 msgr2=0x7f394c0c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f394c04c480 tx=0x7f3940017000 comp rx=0 tx=0).stop 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 shutdown_connections 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f39380604c0 0x7f3938062970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f394c0ca650 0x7f394c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c159bb0 0x7f394c0c5e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 --2- 172.21.15.7:0/2138145578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f394c155840 0x7f394c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 >> 172.21.15.7:0/2138145578 conn(0x7f394c06f380 msgr2=0x7f394c154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 shutdown_connections 2023-10-06T11:21:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.731+0000 7f3952fbe700 1 -- 172.21.15.7:0/2138145578 wait complete. 2023-10-06T11:21:53.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T11:21:53.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:21:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 Processor -- start 2023-10-06T11:21:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 -- start start 2023-10-06T11:21:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36d406bfb0 0x7f36d40c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155c30 0x7f36d40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d40c9a60 con 0x7f36d4155860 2023-10-06T11:21:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d40caea0 con 0x7f36d406bfb0 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36da0a2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d40c9c90 con 0x7f36d4155c30 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d90a0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f36d4155860 0x7f36d406b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55142/0 (socket says 172.21.15.7:55142) 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d90a0700 1 -- 172.21.15.7:0/3708333842 learned_addr learned my addr 172.21.15.7:0/3708333842 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 --2- 172.21.15.7:0/3708333842 >> v2:172.21.15.7:3300/0 conn(0x7f36d406bfb0 0x7f36d40c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d98a1700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155c30 0x7f36d40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d3fff700 1 -- 172.21.15.7:0/3708333842 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913807558 0 0) 0x7f36d40c9a60 con 0x7f36d4155860 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d3fff700 1 -- 172.21.15.7:0/3708333842 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36bc003570 con 0x7f36d4155860 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 -- 172.21.15.7:0/3708333842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155c30 msgr2=0x7f36d40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155c30 0x7f36d40c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 -- 172.21.15.7:0/3708333842 >> v1:172.21.15.7:6789/0 conn(0x7f36d4155860 legacy=0x7f36d406b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 -- 172.21.15.7:0/3708333842 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40ca570 con 0x7f36d406bfb0 2023-10-06T11:21:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d889f700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d40c8500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36c80098e0 tx=0x7f36c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7aa7c28b50c860 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d3fff700 1 -- 172.21.15.7:0/3708333842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800b640 con 0x7f36d406bfb0 2023-10-06T11:21:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d3fff700 1 -- 172.21.15.7:0/3708333842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c800b7a0 con 0x7f36d406bfb0 2023-10-06T11:21:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.847+0000 7f36d3fff700 1 -- 172.21.15.7:0/3708333842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800bb40 con 0x7f36d406bfb0 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- 172.21.15.7:0/3708333842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 msgr2=0x7f36d40c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d40c8500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36c80098e0 tx=0x7f36c80091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- 172.21.15.7:0/3708333842 shutdown_connections 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155c30 0x7f36d40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- 172.21.15.7:0/3708333842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d40c8500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- 172.21.15.7:0/3708333842 >> 172.21.15.7:0/3708333842 conn(0x7f36d406f2a0 msgr2=0x7f36d4068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- 172.21.15.7:0/3708333842 shutdown_connections 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- 172.21.15.7:0/3708333842 wait complete. 2023-10-06T11:21:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 Processor -- start 2023-10-06T11:21:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- start start 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155860 0x7f36d40cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 0x7f36d40cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d40c5250 con 0x7f36d406bfb0 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4167040 con 0x7f36d4155c30 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36da0a2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d40c4920 con 0x7f36d4155860 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55056/0 (socket says 172.21.15.7:55056) 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 -- 172.21.15.7:0/2629249005 learned_addr learned my addr 172.21.15.7:0/2629249005 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d889f700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155860 0x7f36d40cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d98a1700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 0x7f36d40cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 -- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155860 msgr2=0x7f36d40cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155860 0x7f36d40cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 -- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 msgr2=0x7f36d40cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 0x7f36d40cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d90a0700 1 -- 172.21.15.7:0/2629249005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8009420 con 0x7f36d406bfb0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.851+0000 7f36d98a1700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 0x7f36d40cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d90a0700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9a0 tx=0x7f36c4007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4005b30 con 0x7f36d406bfb0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c4005c90 con 0x7f36d406bfb0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d40c4c00 con 0x7f36d406bfb0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c400ca30 con 0x7f36d406bfb0 2023-10-06T11:21:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d414c470 con 0x7f36d406bfb0 2023-10-06T11:21:53.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d406da60 con 0x7f36d406bfb0 2023-10-06T11:21:53.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f36c40070f0 con 0x7f36d406bfb0 2023-10-06T11:21:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.855+0000 7f36d17fa700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 0x7f36c0062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.859+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f36c407e8d0 con 0x7f36d406bfb0 2023-10-06T11:21:53.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.863+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f36c404f190 con 0x7f36d406bfb0 2023-10-06T11:21:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.891+0000 7f36d889f700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 0x7f36c0062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:53.891+0000 7f36d889f700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 0x7f36c0062970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36d406c3b0 tx=0x7f36c8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.027+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36d404b5c0 con 0x7f36d406bfb0 2023-10-06T11:21:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.027+0000 7f36d17fa700 1 -- 172.21.15.7:0/2629249005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36c40527b0 con 0x7f36d406bfb0 2023-10-06T11:21:54.033 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 msgr2=0x7f36c0062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 0x7f36c0062970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36d406c3b0 tx=0x7f36c8017000 comp rx=0 tx=0).stop 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 msgr2=0x7f36d414bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9a0 tx=0x7f36c4007e90 comp rx=0 tx=0).stop 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 shutdown_connections 2023-10-06T11:21:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f36c00604c0 0x7f36c0062970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36d4155c30 0x7f36d40cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36d4155860 0x7f36d40cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 --2- 172.21.15.7:0/2629249005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d406bfb0 0x7f36d414bf30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 >> 172.21.15.7:0/2629249005 conn(0x7f36d406f2a0 msgr2=0x7f36d414e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 shutdown_connections 2023-10-06T11:21:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.031+0000 7f36da0a2700 1 -- 172.21.15.7:0/2629249005 wait complete. 2023-10-06T11:21:54.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:21:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 Processor -- start 2023-10-06T11:21:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 -- start start 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52c415c3e0 0x7f52c40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52c406cfd0 0x7f52c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415ca20 0x7f52c415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40cca80 con 0x7f52c415c3e0 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40cba50 con 0x7f52c406cfd0 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.159+0000 7f52c9981700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c406d990 con 0x7f52c415ca20 2023-10-06T11:21:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52c415c3e0 0x7f52c40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52c415c3e0 0x7f52c40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55068/0 (socket says 172.21.15.7:55068) 2023-10-06T11:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c3fff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52c406cfd0 0x7f52c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9180700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415ca20 0x7f52c415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 -- 172.21.15.7:0/2949996890 learned_addr learned my addr 172.21.15.7:0/2949996890 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 -- 172.21.15.7:0/2949996890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415ca20 msgr2=0x7f52c415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 --2- 172.21.15.7:0/2949996890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415ca20 0x7f52c415cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 -- 172.21.15.7:0/2949996890 >> v2:172.21.15.50:3300/0 conn(0x7f52c406cfd0 msgr2=0x7f52c406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 --2- 172.21.15.7:0/2949996890 >> v2:172.21.15.50:3300/0 conn(0x7f52c406cfd0 0x7f52c406d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 -- 172.21.15.7:0/2949996890 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40c84f0 con 0x7f52c415c3e0 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c897f700 1 --2- 172.21.15.7:0/2949996890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415c3e0 0x7f52c40cae70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52c406de50 tx=0x7f52ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235080f46744540 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c37fe700 1 -- 172.21.15.7:0/2949996890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00b640 con 0x7f52c415c3e0 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c37fe700 1 -- 172.21.15.7:0/2949996890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00b7a0 con 0x7f52c415c3e0 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c37fe700 1 -- 172.21.15.7:0/2949996890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00bb40 con 0x7f52c415c3e0 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 -- 172.21.15.7:0/2949996890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415c3e0 msgr2=0x7f52c40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 --2- 172.21.15.7:0/2949996890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415c3e0 0x7f52c40cae70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52c406de50 tx=0x7f52ac009250 comp rx=0 tx=0).stop 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 -- 172.21.15.7:0/2949996890 shutdown_connections 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 --2- 172.21.15.7:0/2949996890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415ca20 0x7f52c415cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 --2- 172.21.15.7:0/2949996890 >> v2:172.21.15.50:3300/0 conn(0x7f52c406cfd0 0x7f52c406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 --2- 172.21.15.7:0/2949996890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415c3e0 0x7f52c40cae70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 -- 172.21.15.7:0/2949996890 >> 172.21.15.7:0/2949996890 conn(0x7f52c40ce570 msgr2=0x7f52c40ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 -- 172.21.15.7:0/2949996890 shutdown_connections 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.163+0000 7f52c9981700 1 -- 172.21.15.7:0/2949996890 wait complete. 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 Processor -- start 2023-10-06T11:21:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 -- start start 2023-10-06T11:21:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 0x7f52c406a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415c3e0 0x7f52c4141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4069440 con 0x7f52c415ca20 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4069720 con 0x7f52c406cfd0 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9981700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4069a00 con 0x7f52c415c3e0 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55070/0 (socket says 172.21.15.7:55070) 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 -- 172.21.15.7:0/2595698200 learned_addr learned my addr 172.21.15.7:0/2595698200 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c897f700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 0x7f52c406a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c3fff700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415c3e0 0x7f52c4141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 -- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415c3e0 msgr2=0x7f52c4141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415c3e0 0x7f52c4141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 -- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 msgr2=0x7f52c406a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 0x7f52c406a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 -- 172.21.15.7:0/2595698200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0094f0 con 0x7f52c415ca20 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c897f700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 0x7f52c406a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.167+0000 7f52c9180700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e90 tx=0x7f52bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00cb70 con 0x7f52c415ca20 2023-10-06T11:21:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00f040 con 0x7f52c415ca20 2023-10-06T11:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c4069ce0 con 0x7f52c415ca20 2023-10-06T11:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc014480 con 0x7f52c415ca20 2023-10-06T11:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41419c0 con 0x7f52c415ca20 2023-10-06T11:21:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.171+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c406eb50 con 0x7f52c415ca20 2023-10-06T11:21:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.175+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f52bc014830 con 0x7f52c415ca20 2023-10-06T11:21:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.175+0000 7f52c0ff9700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 0x7f52a4062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.175+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f52bc07d100 con 0x7f52c415ca20 2023-10-06T11:21:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.175+0000 7f52c897f700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 0x7f52a4062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.175+0000 7f52c897f700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 0x7f52a4062970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52c406de50 tx=0x7f52ac017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.183+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52bc04d9c0 con 0x7f52c415ca20 2023-10-06T11:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52c404b5c0 con 0x7f52c415ca20 2023-10-06T11:21:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c0ff9700 1 -- 172.21.15.7:0/2595698200 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52bc050fe0 con 0x7f52c415ca20 2023-10-06T11:21:54.356 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 msgr2=0x7f52a4062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 0x7f52a4062970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52c406de50 tx=0x7f52ac017000 comp rx=0 tx=0).stop 2023-10-06T11:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 msgr2=0x7f52c4154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e90 tx=0x7f52bc00b040 comp rx=0 tx=0).stop 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 shutdown_connections 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f52a40604c0 0x7f52a4062970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c415ca20 0x7f52c4154f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f52c415c3e0 0x7f52c4141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 --2- 172.21.15.7:0/2595698200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c406cfd0 0x7f52c406a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 >> 172.21.15.7:0/2595698200 conn(0x7f52c40ce570 msgr2=0x7f52c4155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 shutdown_connections 2023-10-06T11:21:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.351+0000 7f52c9981700 1 -- 172.21.15.7:0/2595698200 wait complete. 2023-10-06T11:21:54.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:21:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 Processor -- start 2023-10-06T11:21:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 -- start start 2023-10-06T11:21:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe8e006c1f0 0x7fe8e01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e00c80a0 con 0x7fe8e006be20 2023-10-06T11:21:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e015d470 con 0x7fe8e006d2f0 2023-10-06T11:21:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8dffff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e015d750 con 0x7fe8e006c1f0 2023-10-06T11:21:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8deffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8e006be20 0x7fe8e015cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55158/0 (socket says 172.21.15.7:55158) 2023-10-06T11:21:54.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8deffd700 1 -- 172.21.15.7:0/3534257231 learned_addr learned my addr 172.21.15.7:0/3534257231 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:54.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.475+0000 7fe8df7fe700 1 --2- 172.21.15.7:0/3534257231 >> v2:172.21.15.50:3300/0 conn(0x7fe8e006c1f0 0x7fe8e01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201894165 0 0) 0x7fe8e00c80a0 con 0x7fe8e006be20 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c8003570 con 0x7fe8e006be20 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3222582264 0 0) 0x7fe8c8003570 con 0x7fe8e006be20 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3534257231 >> v1:172.21.15.50:6789/0 conn(0x7fe8e006d2f0 legacy=0x7fe8e00ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3534257231 >> v1:172.21.15.7:6789/0 conn(0x7fe8e006be20 legacy=0x7fe8e015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3534257231 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e015da30 con 0x7fe8e006c1f0 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8df7fe700 1 --2- 172.21.15.7:0/3534257231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e01589a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009e60 tx=0x7fe8d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bde343942f4e35c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800e070 con 0x7fe8e006c1f0 2023-10-06T11:21:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d800b850 con 0x7fe8e006c1f0 2023-10-06T11:21:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8ddffb700 1 -- 172.21.15.7:0/3534257231 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0001470 con 0x7fe8e006c1f0 2023-10-06T11:21:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- 172.21.15.7:0/3534257231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 msgr2=0x7fe8e01589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3534257231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e01589a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009e60 tx=0x7fe8d8009780 comp rx=0 tx=0).stop 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- 172.21.15.7:0/3534257231 shutdown_connections 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3534257231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e01589a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- 172.21.15.7:0/3534257231 >> 172.21.15.7:0/3534257231 conn(0x7fe8e00cdb00 msgr2=0x7fe8e00cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- 172.21.15.7:0/3534257231 shutdown_connections 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- 172.21.15.7:0/3534257231 wait complete. 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 Processor -- start 2023-10-06T11:21:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.479+0000 7fe8dffff700 1 -- start start 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe8e006be20 0x7fe8e0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e00af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00b19f0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00b1560 con 0x7fe8e006c1f0 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00b0950 con 0x7fe8e006be20 2023-10-06T11:21:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55072/0 (socket says 172.21.15.7:55072) 2023-10-06T11:21:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3184857865 learned_addr learned my addr 172.21.15.7:0/3184857865 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8deffd700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe8e006be20 0x7fe8e0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8de7fc700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe8e006be20 msgr2=0x7fe8e0155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe8e006be20 0x7fe8e0155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 msgr2=0x7fe8e00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e00af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 -- 172.21.15.7:0/3184857865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8009a20 con 0x7fe8e006d2f0 2023-10-06T11:21:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8df7fe700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097c0 tx=0x7fe8d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8012c40 con 0x7fe8e006d2f0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d8012da0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e00b0c30 con 0x7fe8e006d2f0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d801e680 con 0x7fe8e006d2f0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.483+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e00b11f0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e00b16f0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fe8d801e9f0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8c6ffd700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 0x7fe8cc062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe8d807e8e0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8deffd700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 0x7fe8cc062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.487+0000 7fe8deffd700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 0x7fe8cc062970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00011e0 tx=0x7fe8d0000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.495+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe8d804f1a0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.531+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe8d804f570 con 0x7fe8e006d2f0 2023-10-06T11:21:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.667+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe8e004b5c0 con 0x7fe8e006d2f0 2023-10-06T11:21:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.667+0000 7fe8c6ffd700 1 -- 172.21.15.7:0/3184857865 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe8d8026400 con 0x7fe8e006d2f0 2023-10-06T11:21:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:21:54.672 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": "rbd" 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 2, 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:21:54.673 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.667+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 msgr2=0x7fe8cc062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.667+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 0x7fe8cc062970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00011e0 tx=0x7fe8d0000bc0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.667+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 msgr2=0x7fe8e00af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097c0 tx=0x7fe8d8017000 comp rx=0 tx=0).stop 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 shutdown_connections 2023-10-06T11:21:54.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe8cc0604c0 0x7fe8cc062970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e006d2f0 0x7fe8e00af780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e006c1f0 0x7fe8e00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 --2- 172.21.15.7:0/3184857865 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe8e006be20 0x7fe8e0155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 >> 172.21.15.7:0/3184857865 conn(0x7fe8e00cdb00 msgr2=0x7fe8e00cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 shutdown_connections 2023-10-06T11:21:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.671+0000 7fe8dffff700 1 -- 172.21.15.7:0/3184857865 wait complete. 2023-10-06T11:21:54.686 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:21:54.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:21:54.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 Processor -- start 2023-10-06T11:21:54.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 -- start start 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 0x7f535c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0c8c30 con 0x7f535c155bf0 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c15a710 con 0x7f535c069650 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f53647c1700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0c8300 con 0x7f535c155fc0 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f536255d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f535c155bf0 0x7f535c068f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55160/0 (socket says 172.21.15.7:55160) 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f536255d700 1 -- 172.21.15.7:0/1517908283 learned_addr learned my addr 172.21.15.7:0/1517908283 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:54.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f5362d5e700 1 --2- 172.21.15.7:0/1517908283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 0x7f535c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.791+0000 7f536155b700 1 -- 172.21.15.7:0/1517908283 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824326470 0 0) 0x7f535c15a710 con 0x7f535c069650 2023-10-06T11:21:54.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f536155b700 1 -- 172.21.15.7:0/1517908283 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5348003570 con 0x7f535c069650 2023-10-06T11:21:54.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f5362d5e700 1 -- 172.21.15.7:0/1517908283 >> v1:172.21.15.50:6789/0 conn(0x7f535c069650 legacy=0x7f535c1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f5362d5e700 1 -- 172.21.15.7:0/1517908283 >> v1:172.21.15.7:6789/0 conn(0x7f535c155bf0 legacy=0x7f535c068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f5362d5e700 1 -- 172.21.15.7:0/1517908283 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0c85e0 con 0x7f535c155fc0 2023-10-06T11:21:54.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f5362d5e700 1 --2- 172.21.15.7:0/1517908283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 0x7f535c15a0d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5354009e90 tx=0x7f5354009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e12cd6fd09012e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f536155b700 1 -- 172.21.15.7:0/1517908283 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400e070 con 0x7f535c155fc0 2023-10-06T11:21:54.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f536155b700 1 -- 172.21.15.7:0/1517908283 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400bc30 con 0x7f535c155fc0 2023-10-06T11:21:54.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f536155b700 1 -- 172.21.15.7:0/1517908283 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354012430 con 0x7f535c155fc0 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- 172.21.15.7:0/1517908283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 msgr2=0x7f535c15a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 --2- 172.21.15.7:0/1517908283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 0x7f535c15a0d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5354009e90 tx=0x7f5354009780 comp rx=0 tx=0).stop 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- 172.21.15.7:0/1517908283 shutdown_connections 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 --2- 172.21.15.7:0/1517908283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155fc0 0x7f535c15a0d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- 172.21.15.7:0/1517908283 >> 172.21.15.7:0/1517908283 conn(0x7f535c06ea40 msgr2=0x7f535c06ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- 172.21.15.7:0/1517908283 shutdown_connections 2023-10-06T11:21:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- 172.21.15.7:0/1517908283 wait complete. 2023-10-06T11:21:54.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 Processor -- start 2023-10-06T11:21:54.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- start start 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 0x7f535c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f535c155fc0 0x7f535c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0c2680 con 0x7f535c155fc0 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0c2960 con 0x7f535c069650 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.795+0000 7f53647c1700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0c2c40 con 0x7f535c155bf0 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f536255d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5362d5e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f535c155fc0 0x7f535c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f536255d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:49818/0 (socket says 172.21.15.7:49818) 2023-10-06T11:21:54.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 0x7f535c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f536255d700 1 -- 172.21.15.7:0/3565360098 learned_addr learned my addr 172.21.15.7:0/3565360098 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 -- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 msgr2=0x7f535c0c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 -- 172.21.15.7:0/3565360098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f535c155fc0 msgr2=0x7f535c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f535c155fc0 0x7f535c156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 -- 172.21.15.7:0/3565360098 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354009a20 con 0x7f535c155bf0 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f536255d700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f5361d5c700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 0x7f535c14c840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f535c069a50 tx=0x7f534c007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011590 con 0x7f535c155bf0 2023-10-06T11:21:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c0116f0 con 0x7f535c155bf0 2023-10-06T11:21:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011ab0 con 0x7f535c155bf0 2023-10-06T11:21:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c0c2f20 con 0x7f535c155bf0 2023-10-06T11:21:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c0c3720 con 0x7f535c155bf0 2023-10-06T11:21:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.799+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c0c3420 con 0x7f535c155bf0 2023-10-06T11:21:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.803+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f534c01c460 con 0x7f535c155bf0 2023-10-06T11:21:54.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.803+0000 7f53527fc700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 0x7f5340062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:54.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.803+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f534c07c120 con 0x7f535c155bf0 2023-10-06T11:21:54.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.803+0000 7f536255d700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 0x7f5340062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:54.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.803+0000 7f536255d700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 0x7f5340062a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5358004900 tx=0x7f5358007f70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.811+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f534c04c900 con 0x7f535c155bf0 2023-10-06T11:21:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:54.971+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f535c04be00 con 0x7f535c155bf0 2023-10-06T11:21:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.539+0000 7f53527fc700 1 -- 172.21.15.7:0/3565360098 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f534c04ff20 con 0x7f535c155bf0 2023-10-06T11:21:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 msgr2=0x7f5340062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 0x7f5340062a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5358004900 tx=0x7f5358007f70 comp rx=0 tx=0).stop 2023-10-06T11:21:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 msgr2=0x7f535c14c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 0x7f535c14c840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f535c069a50 tx=0x7f534c007d50 comp rx=0 tx=0).stop 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 shutdown_connections 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5340060580 0x7f5340062a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f535c155fc0 0x7f535c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f535c155bf0 0x7f535c14c840 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 --2- 172.21.15.7:0/3565360098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f535c069650 0x7f535c0c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 >> 172.21.15.7:0/3565360098 conn(0x7f535c06ea40 msgr2=0x7f535c1598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 shutdown_connections 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.547+0000 7f53647c1700 1 -- 172.21.15.7:0/3565360098 wait complete. 2023-10-06T11:21:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-10-06T11:21:55.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-06T11:21:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.679+0000 7f605bfff700 1 Processor -- start 2023-10-06T11:21:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.679+0000 7f605bfff700 1 -- start start 2023-10-06T11:21:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f605c15c3e0 0x7f605c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 0x7f605c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c0c84f0 con 0x7f605c15ca20 2023-10-06T11:21:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c0cca80 con 0x7f605c15c3e0 2023-10-06T11:21:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c0c9970 con 0x7f605c06cfd0 2023-10-06T11:21:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f605c15c3e0 0x7f605c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f605c15ca20 0x7f605c15ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55174/0 (socket says 172.21.15.7:55174) 2023-10-06T11:21:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605b7fe700 1 -- 172.21.15.7:0/1434493468 learned_addr learned my addr 172.21.15.7:0/1434493468 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f6053fff700 1 --2- 172.21.15.7:0/1434493468 >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 0x7f605c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 -- 172.21.15.7:0/1434493468 >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 msgr2=0x7f605c06d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 --2- 172.21.15.7:0/1434493468 >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 0x7f605c06d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 -- 172.21.15.7:0/1434493468 >> v1:172.21.15.7:6789/0 conn(0x7f605c15ca20 legacy=0x7f605c15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605a7fc700 1 -- 172.21.15.7:0/1434493468 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088214294 0 0) 0x7f605c0c84f0 con 0x7f605c15ca20 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 -- 172.21.15.7:0/1434493468 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c15c7b0 con 0x7f605c15c3e0 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f6053fff700 1 --2- 172.21.15.7:0/1434493468 >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 0x7f605c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605affd700 1 --2- 172.21.15.7:0/1434493468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c0cae70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f605c06de50 tx=0x7f604c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08bfeac4bccf7b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605a7fc700 1 -- 172.21.15.7:0/1434493468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c00b6b0 con 0x7f605c15c3e0 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605a7fc700 1 -- 172.21.15.7:0/1434493468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c00b810 con 0x7f605c15c3e0 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605a7fc700 1 -- 172.21.15.7:0/1434493468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c00bbb0 con 0x7f605c15c3e0 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- 172.21.15.7:0/1434493468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 msgr2=0x7f605c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 --2- 172.21.15.7:0/1434493468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c0cae70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f605c06de50 tx=0x7f604c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- 172.21.15.7:0/1434493468 shutdown_connections 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 --2- 172.21.15.7:0/1434493468 >> v2:172.21.15.50:3300/0 conn(0x7f605c06cfd0 0x7f605c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 --2- 172.21.15.7:0/1434493468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c0cae70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- 172.21.15.7:0/1434493468 >> 172.21.15.7:0/1434493468 conn(0x7f605c0ce570 msgr2=0x7f605c0ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- 172.21.15.7:0/1434493468 shutdown_connections 2023-10-06T11:21:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.683+0000 7f605bfff700 1 -- 172.21.15.7:0/1434493468 wait complete. 2023-10-06T11:21:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.687+0000 7f605bfff700 1 Processor -- start 2023-10-06T11:21:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.687+0000 7f605bfff700 1 -- start start 2023-10-06T11:21:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.687+0000 7f605bfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 0x7f605c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 0x7f605c06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c15fe40 con 0x7f605c15c3e0 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c15d370 con 0x7f605c15ca20 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c15d630 con 0x7f605c06cfd0 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55094/0 (socket says 172.21.15.7:55094) 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 -- 172.21.15.7:0/2736673036 learned_addr learned my addr 172.21.15.7:0/2736673036 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605affd700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 0x7f605c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605b7fe700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 0x7f605c06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 -- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 msgr2=0x7f605c154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 0x7f605c154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 -- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 msgr2=0x7f605c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 0x7f605c06a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 -- 172.21.15.7:0/2736673036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c009420 con 0x7f605c15c3e0 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605b7fe700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 0x7f605c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605affd700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 0x7f605c154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:55.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6053fff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604400e8d0 tx=0x7f6044008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400ee70 con 0x7f605c15c3e0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604400c5e0 con 0x7f605c15c3e0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c0b21e0 con 0x7f605c15c3e0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400c980 con 0x7f605c15c3e0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c0b27a0 con 0x7f605c15c3e0 2023-10-06T11:21:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.691+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c06eb50 con 0x7f605c15c3e0 2023-10-06T11:21:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.695+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f604400cd30 con 0x7f605c15c3e0 2023-10-06T11:21:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.695+0000 7f6052ffd700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 0x7f6040062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.695+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f604407c5f0 con 0x7f605c15c3e0 2023-10-06T11:21:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.695+0000 7f605affd700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 0x7f6040062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.695+0000 7f605affd700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 0x7f6040062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f604c0093f0 tx=0x7f604c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.707+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f604404cc60 con 0x7f605c15c3e0 2023-10-06T11:21:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:55.871+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f605c04b5c0 con 0x7f605c15c3e0 2023-10-06T11:21:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.551+0000 7f6052ffd700 1 -- 172.21.15.7:0/2736673036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f6044050280 con 0x7f605c15c3e0 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 msgr2=0x7f6040062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 0x7f6040062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f604c0093f0 tx=0x7f604c017000 comp rx=0 tx=0).stop 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 msgr2=0x7f605c069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604400e8d0 tx=0x7f6044008040 comp rx=0 tx=0).stop 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 shutdown_connections 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6040060550 0x7f6040062a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605c15ca20 0x7f605c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f605c15c3e0 0x7f605c069f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 --2- 172.21.15.7:0/2736673036 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f605c06cfd0 0x7f605c154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 >> 172.21.15.7:0/2736673036 conn(0x7f605c0ce570 msgr2=0x7f605c15a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 shutdown_connections 2023-10-06T11:21:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.575+0000 7f605bfff700 1 -- 172.21.15.7:0/2736673036 wait complete. 2023-10-06T11:21:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-10-06T11:21:56.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:21:56.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.695+0000 7fd4e6917700 1 Processor -- start 2023-10-06T11:21:56.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.695+0000 7fd4e6917700 1 -- start start 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6917700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4e00ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6917700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd4e0159f20 0x7fd4e00cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6917700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e015d320 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6917700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e00cd060 con 0x7fd4e0159f20 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6917700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015a390 con 0x7fd4e00b21d0 2023-10-06T11:21:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6116700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4e00ccbe0 0x7fd4e006d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55186/0 (socket says 172.21.15.7:55186) 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6116700 1 -- 172.21.15.7:0/3748054933 learned_addr learned my addr 172.21.15.7:0/3748054933 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5915700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4e00ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 --2- 172.21.15.7:0/3748054933 >> v2:172.21.15.50:3300/0 conn(0x7fd4e0159f20 0x7fd4e00cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901675417 0 0) 0x7fd4e015d320 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc003570 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118394081 0 0) 0x7fd4cc003570 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 msgr2=0x7fd4e00ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4e00ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 >> v2:172.21.15.50:3300/0 conn(0x7fd4e0159f20 msgr2=0x7fd4e00cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 --2- 172.21.15.7:0/3748054933 >> v2:172.21.15.50:3300/0 conn(0x7fd4e0159f20 0x7fd4e00cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00ce940 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7fd4d80017f0 con 0x7fd4e00ccbe0 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 >> v1:172.21.15.7:6789/0 conn(0x7fd4e00ccbe0 legacy=0x7fd4e006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4cc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4cc003fa0 0x7fd4cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00cd060 con 0x7fd4cc003fa0 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015a390 con 0x7fd4cc004d50 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00ba60 con 0x7fd4e00b21d0 2023-10-06T11:21:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4cc003fa0 0x7fd4cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5915700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4cc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6116700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 -- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 msgr2=0x7fd4cc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e00b21d0 0x7fd4cc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 -- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc004d50 msgr2=0x7fd4cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 -- 172.21.15.7:0/3748054933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc003570 con 0x7fd4cc003fa0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e6116700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.699+0000 7fd4e5114700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4cc003fa0 0x7fd4cc004810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009c00 tx=0x7fd4dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267038a9896fc839 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00e710 con 0x7fd4cc003fa0 2023-10-06T11:21:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00e870 con 0x7fd4cc003fa0 2023-10-06T11:21:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e4913700 1 -- 172.21.15.7:0/3748054933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00ec30 con 0x7fd4cc003fa0 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 -- 172.21.15.7:0/3748054933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4cc003fa0 msgr2=0x7fd4cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 --2- 172.21.15.7:0/3748054933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4cc003fa0 0x7fd4cc004810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009c00 tx=0x7fd4dc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6116700 1 -- 172.21.15.7:0/3748054933 reap_dead start 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 -- 172.21.15.7:0/3748054933 shutdown_connections 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 -- 172.21.15.7:0/3748054933 >> 172.21.15.7:0/3748054933 conn(0x7fd4e0067e30 msgr2=0x7fd4e0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 -- 172.21.15.7:0/3748054933 shutdown_connections 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.703+0000 7fd4e6917700 1 -- 172.21.15.7:0/3748054933 wait complete. 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 Processor -- start 2023-10-06T11:21:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 -- start start 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc003fa0 0x7fd4e015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e01690b0 0x7fd4e016ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015c150 con 0x7fd4e0168c40 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015b540 con 0x7fd4cc003fa0 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6917700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015b820 con 0x7fd4e01690b0 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55118/0 (socket says 172.21.15.7:55118) 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 -- 172.21.15.7:0/2645114422 learned_addr learned my addr 172.21.15.7:0/2645114422 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5915700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc003fa0 0x7fd4e015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e6116700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e01690b0 0x7fd4e016ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 -- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e01690b0 msgr2=0x7fd4e016ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e01690b0 0x7fd4e016ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 -- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc003fa0 msgr2=0x7fd4e015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc003fa0 0x7fd4e015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 -- 172.21.15.7:0/2645114422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e015bb00 con 0x7fd4e0168c40 2023-10-06T11:21:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.707+0000 7fd4e5114700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0046a0 tx=0x7fd4dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc022070 con 0x7fd4e0168c40 2023-10-06T11:21:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00e9e0 con 0x7fd4e0168c40 2023-10-06T11:21:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e015c150 con 0x7fd4e0168c40 2023-10-06T11:21:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc01c450 con 0x7fd4e0168c40 2023-10-06T11:21:56.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e00b3150 con 0x7fd4e0168c40 2023-10-06T11:21:56.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e00ce030 con 0x7fd4e0168c40 2023-10-06T11:21:56.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd4dc01c880 con 0x7fd4e0168c40 2023-10-06T11:21:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 0x7fd4c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd4dc07df30 con 0x7fd4e0168c40 2023-10-06T11:21:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.711+0000 7fd4e5915700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 0x7fd4c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.715+0000 7fd4e5915700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 0x7fd4c4062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009e00 tx=0x7fd4d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.719+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd4e00ce030 con 0x7fd4e0168c40 2023-10-06T11:21:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:56.919+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd4e00ce030 con 0x7fd4e0168c40 2023-10-06T11:21:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.571+0000 7fd4d1ffb700 1 -- 172.21.15.7:0/2645114422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd4e00ce030 con 0x7fd4e0168c40 2023-10-06T11:21:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 msgr2=0x7fd4c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 0x7fd4c4062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009e00 tx=0x7fd4d4008040 comp rx=0 tx=0).stop 2023-10-06T11:21:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 msgr2=0x7fd4e00b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0046a0 tx=0x7fd4dc01a000 comp rx=0 tx=0).stop 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 shutdown_connections 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd4c4060550 0x7fd4c4062a00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd4e01690b0 0x7fd4e016ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e0168c40 0x7fd4e00b2c10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 --2- 172.21.15.7:0/2645114422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4cc003fa0 0x7fd4e015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 >> 172.21.15.7:0/2645114422 conn(0x7fd4e0067e30 msgr2=0x7fd4e00c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 shutdown_connections 2023-10-06T11:21:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.583+0000 7fd4e6917700 1 -- 172.21.15.7:0/2645114422 wait complete. 2023-10-06T11:21:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T11:21:57.600 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-06T11:21:57.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-06T11:21:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.703+0000 7f78f9a9c700 1 Processor -- start 2023-10-06T11:21:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.703+0000 7f78f9a9c700 1 -- start start 2023-10-06T11:21:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f9a9c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40afe60 0x7f78f40b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f9a9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78f40cc2b0 0x7f78f40c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f9a9c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f40c9180 con 0x7f78f40cc830 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f9a9c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f40c8b70 con 0x7f78f40cc2b0 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f9a9c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40c40b0 con 0x7f78f40afe60 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78f40cc2b0 0x7f78f40c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78f40cc2b0 0x7f78f40c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55134/0 (socket says 172.21.15.7:55134) 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 -- 172.21.15.7:0/2136266603 learned_addr learned my addr 172.21.15.7:0/2136266603 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f8a9a700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40afe60 0x7f78f40b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 -- 172.21.15.7:0/2136266603 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40afe60 msgr2=0x7f78f40b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:21:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40afe60 0x7f78f40b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 -- 172.21.15.7:0/2136266603 >> v1:172.21.15.7:6789/0 conn(0x7f78f40cc830 legacy=0x7f78f40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 -- 172.21.15.7:0/2136266603 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40c8e10 con 0x7f78f40cc2b0 2023-10-06T11:21:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f929b700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40cc2b0 0x7f78f40c2ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009f20 tx=0x7f78ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9501b7a0f77f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f37fe700 1 -- 172.21.15.7:0/2136266603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00e070 con 0x7f78f40cc2b0 2023-10-06T11:21:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f37fe700 1 -- 172.21.15.7:0/2136266603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b9d0 con 0x7f78f40cc2b0 2023-10-06T11:21:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.707+0000 7f78f37fe700 1 -- 172.21.15.7:0/2136266603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0092e0 con 0x7f78f40cc2b0 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- 172.21.15.7:0/2136266603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40cc2b0 msgr2=0x7f78f40c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40cc2b0 0x7f78f40c2ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009f20 tx=0x7f78ec009840 comp rx=0 tx=0).stop 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- 172.21.15.7:0/2136266603 shutdown_connections 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40cc2b0 0x7f78f40c2ac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/2136266603 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40afe60 0x7f78f40b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- 172.21.15.7:0/2136266603 >> 172.21.15.7:0/2136266603 conn(0x7f78f4087310 msgr2=0x7f78f40cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- 172.21.15.7:0/2136266603 shutdown_connections 2023-10-06T11:21:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- 172.21.15.7:0/2136266603 wait complete. 2023-10-06T11:21:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 Processor -- start 2023-10-06T11:21:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.711+0000 7f78f9a9c700 1 -- start start 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78f40cc2b0 0x7f78f40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40cc830 0x7f78f415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40c4e70 con 0x7f78f40afe60 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4153180 con 0x7f78f40cc2b0 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40b1670 con 0x7f78f40cc830 2023-10-06T11:21:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55146/0 (socket says 172.21.15.7:55146) 2023-10-06T11:21:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 -- 172.21.15.7:0/3264656832 learned_addr learned my addr 172.21.15.7:0/3264656832 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f929b700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40cc830 0x7f78f415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f3fff700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78f40cc2b0 0x7f78f40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 -- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40cc830 msgr2=0x7f78f415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40cc830 0x7f78f415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 -- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78f40cc2b0 msgr2=0x7f78f40b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78f40cc2b0 0x7f78f40b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 -- 172.21.15.7:0/3264656832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009ae0 con 0x7f78f40afe60 2023-10-06T11:21:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f8a9a700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e930 tx=0x7f78e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e400eed0 con 0x7f78f40afe60 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e400c850 con 0x7f78f40afe60 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f40c50e0 con 0x7f78f40afe60 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e400cbf0 con 0x7f78f40afe60 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.715+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f40b19d0 con 0x7f78f40afe60 2023-10-06T11:21:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f406df90 con 0x7f78f40afe60 2023-10-06T11:21:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f78e4013470 con 0x7f78f40afe60 2023-10-06T11:21:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f0ff9700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 0x7f78e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f78e407cbc0 con 0x7f78f40afe60 2023-10-06T11:21:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f3fff700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 0x7f78e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.719+0000 7f78f3fff700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 0x7f78e0062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f78f406b910 tx=0x7f78e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.727+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f78e404cfb0 con 0x7f78f40afe60 2023-10-06T11:21:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:57.899+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f78f40cee70 con 0x7f78f40afe60 2023-10-06T11:21:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.547+0000 7f78f0ff9700 1 -- 172.21.15.7:0/3264656832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f78e40505d0 con 0x7f78f40afe60 2023-10-06T11:21:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 msgr2=0x7f78e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 0x7f78e0062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f78f406b910 tx=0x7f78e8009040 comp rx=0 tx=0).stop 2023-10-06T11:21:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 msgr2=0x7f78f415d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e930 tx=0x7f78e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 shutdown_connections 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f78e0060550 0x7f78e0062a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f78f40cc830 0x7f78f415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78f40cc2b0 0x7f78f40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 --2- 172.21.15.7:0/3264656832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f40afe60 0x7f78f415d770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 >> 172.21.15.7:0/3264656832 conn(0x7f78f4087310 msgr2=0x7f78f40c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 shutdown_connections 2023-10-06T11:21:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.551+0000 7f78f9a9c700 1 -- 172.21.15.7:0/3264656832 wait complete. 2023-10-06T11:21:58.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:21:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.683+0000 7fce61c43700 1 Processor -- start 2023-10-06T11:21:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.683+0000 7fce61c43700 1 -- start start 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fce5c0aee80 0x7fce5c0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c06c790 con 0x7fce5c067e30 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c0c7f30 con 0x7fce5c0aeab0 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c157060 con 0x7fce5c0aee80 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61442700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce5c067e30 0x7fce5c0ac4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55208/0 (socket says 172.21.15.7:55208) 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61442700 1 -- 172.21.15.7:0/129270029 learned_addr learned my addr 172.21.15.7:0/129270029 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:21:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5bfff700 1 --2- 172.21.15.7:0/129270029 >> v2:172.21.15.50:3300/0 conn(0x7fce5c0aee80 0x7fce5c0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624337561 0 0) 0x7fce5c06c790 con 0x7fce5c067e30 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce48003570 con 0x7fce5c067e30 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769575391 0 0) 0x7fce5c0c7f30 con 0x7fce5c0aeab0 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c06c790 con 0x7fce5c0aeab0 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5bfff700 1 -- 172.21.15.7:0/129270029 >> v1:172.21.15.50:6789/0 conn(0x7fce5c0aeab0 legacy=0x7fce5c06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5bfff700 1 -- 172.21.15.7:0/129270029 >> v1:172.21.15.7:6789/0 conn(0x7fce5c067e30 legacy=0x7fce5c0ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5bfff700 1 -- 172.21.15.7:0/129270029 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c06b8a0 con 0x7fce5c0aee80 2023-10-06T11:21:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5bfff700 1 --2- 172.21.15.7:0/129270029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aee80 0x7fce5c0c79f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098e0 tx=0x7fce4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a89a1deb9cab1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011630 con 0x7fce5c0aee80 2023-10-06T11:21:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c011790 con 0x7fce5c0aee80 2023-10-06T11:21:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce5b7fe700 1 -- 172.21.15.7:0/129270029 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011b10 con 0x7fce5c0aee80 2023-10-06T11:21:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- 172.21.15.7:0/129270029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aee80 msgr2=0x7fce5c0c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 --2- 172.21.15.7:0/129270029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aee80 0x7fce5c0c79f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098e0 tx=0x7fce4c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- 172.21.15.7:0/129270029 shutdown_connections 2023-10-06T11:21:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 --2- 172.21.15.7:0/129270029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aee80 0x7fce5c0c79f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- 172.21.15.7:0/129270029 >> 172.21.15.7:0/129270029 conn(0x7fce5c06f2a0 msgr2=0x7fce5c068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:21:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- 172.21.15.7:0/129270029 shutdown_connections 2023-10-06T11:21:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.687+0000 7fce61c43700 1 -- 172.21.15.7:0/129270029 wait complete. 2023-10-06T11:21:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 Processor -- start 2023-10-06T11:21:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 -- start start 2023-10-06T11:21:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aeab0 0x7fce5c15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce5c0aee80 0x7fce5c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c156b90 con 0x7fce5c067e30 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0a45b0 con 0x7fce5c0aeab0 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c166c30 con 0x7fce5c0aee80 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55156/0 (socket says 172.21.15.7:55156) 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 -- 172.21.15.7:0/927486019 learned_addr learned my addr 172.21.15.7:0/927486019 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce5bfff700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aeab0 0x7fce5c15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61442700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce5c0aee80 0x7fce5c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 -- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce5c0aee80 msgr2=0x7fce5c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce5c0aee80 0x7fce5c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 -- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aeab0 msgr2=0x7fce5c15a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aeab0 0x7fce5c15a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 -- 172.21.15.7:0/927486019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c009420 con 0x7fce5c067e30 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce60c41700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce4400c7c0 tx=0x7fce44007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44011680 con 0x7fce5c067e30 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440117e0 con 0x7fce5c067e30 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c166f00 con 0x7fce5c067e30 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.691+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44011ba0 con 0x7fce5c067e30 2023-10-06T11:21:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c15b040 con 0x7fce5c067e30 2023-10-06T11:21:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c06da60 con 0x7fce5c067e30 2023-10-06T11:21:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fce4401c460 con 0x7fce5c067e30 2023-10-06T11:21:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce58ff9700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 0x7fce3c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:21:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fce4407cb30 con 0x7fce5c067e30 2023-10-06T11:21:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.695+0000 7fce5bfff700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 0x7fce3c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:21:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.699+0000 7fce5bfff700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 0x7fce3c062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0093f0 tx=0x7fce4c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:21:58.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.703+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce4404cee0 con 0x7fce5c067e30 2023-10-06T11:21:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:21:58.863+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce5c06bb80 con 0x7fce5c067e30 2023-10-06T11:22:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.607+0000 7fce58ff9700 1 -- 172.21.15.7:0/927486019 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fce44050500 con 0x7fce5c067e30 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 msgr2=0x7fce3c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 0x7fce3c062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0093f0 tx=0x7fce4c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 msgr2=0x7fce5c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce4400c7c0 tx=0x7fce44007e10 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 shutdown_connections 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce3c060550 0x7fce3c062a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce5c0aee80 0x7fce5c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5c0aeab0 0x7fce5c15a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 --2- 172.21.15.7:0/927486019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c067e30 0x7fce5c156650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 >> 172.21.15.7:0/927486019 conn(0x7fce5c06f2a0 msgr2=0x7fce5c0c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:00.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 shutdown_connections 2023-10-06T11:22:00.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.615+0000 7fce61c43700 1 -- 172.21.15.7:0/927486019 wait complete. 2023-10-06T11:22:00.620 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2023-10-06T11:22:00.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-06T11:22:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.731+0000 7f9fae67e700 1 Processor -- start 2023-10-06T11:22:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.731+0000 7f9fae67e700 1 -- start start 2023-10-06T11:22:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 0x7f9fa806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9fa806def0 0x7f9fa8069800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa8069370 con 0x7f9fa80a9900 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa806e330 con 0x7f9fa806def0 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8159c90 con 0x7f9fa8159500 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9fa80a9900 0x7f9fa806e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55224/0 (socket says 172.21.15.7:55224) 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa7fff700 1 -- 172.21.15.7:0/2902684316 learned_addr learned my addr 172.21.15.7:0/2902684316 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9facc1b700 1 --2- 172.21.15.7:0/2902684316 >> v2:172.21.15.50:3300/0 conn(0x7f9fa806def0 0x7f9fa8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2902684316 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 0x7f9fa806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa6ffd700 1 -- 172.21.15.7:0/2902684316 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537483625 0 0) 0x7f9fa8069370 con 0x7f9fa80a9900 2023-10-06T11:22:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa6ffd700 1 -- 172.21.15.7:0/2902684316 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f94003570 con 0x7f9fa80a9900 2023-10-06T11:22:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2902684316 >> v2:172.21.15.50:3300/0 conn(0x7f9fa806def0 msgr2=0x7f9fa8069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2902684316 >> v2:172.21.15.50:3300/0 conn(0x7f9fa806def0 0x7f9fa8069800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2902684316 >> v1:172.21.15.7:6789/0 conn(0x7f9fa80a9900 legacy=0x7f9fa806e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2902684316 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa8159f70 con 0x7f9fa8159500 2023-10-06T11:22:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2902684316 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 0x7f9fa806d9b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098e0 tx=0x7f9f900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4fc7f4e264874a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa6ffd700 1 -- 172.21.15.7:0/2902684316 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011630 con 0x7f9fa8159500 2023-10-06T11:22:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa6ffd700 1 -- 172.21.15.7:0/2902684316 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90011790 con 0x7f9fa8159500 2023-10-06T11:22:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fa6ffd700 1 -- 172.21.15.7:0/2902684316 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011b30 con 0x7f9fa8159500 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- 172.21.15.7:0/2902684316 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 msgr2=0x7f9fa806d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2902684316 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 0x7f9fa806d9b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098e0 tx=0x7f9f900091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- 172.21.15.7:0/2902684316 shutdown_connections 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2902684316 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa8159500 0x7f9fa806d9b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2902684316 >> v2:172.21.15.50:3300/0 conn(0x7f9fa806def0 0x7f9fa8069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- 172.21.15.7:0/2902684316 >> 172.21.15.7:0/2902684316 conn(0x7f9fa8068900 msgr2=0x7f9fa80cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- 172.21.15.7:0/2902684316 shutdown_connections 2023-10-06T11:22:00.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.735+0000 7f9fae67e700 1 -- 172.21.15.7:0/2902684316 wait complete. 2023-10-06T11:22:00.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.739+0000 7f9fae67e700 1 Processor -- start 2023-10-06T11:22:00.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.739+0000 7f9fae67e700 1 -- start start 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.739+0000 7f9fae67e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa806def0 0x7f9fa80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.739+0000 7f9fae67e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fa8159500 0x7f9fa81514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8154140 con 0x7f9fa80a9900 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa81543a0 con 0x7f9fa8159500 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa815c3b0 con 0x7f9fa806def0 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55170/0 (socket says 172.21.15.7:55170) 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2112872827 learned_addr learned my addr 172.21.15.7:0/2112872827 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa7fff700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa806def0 0x7f9fa80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9facc1b700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fa8159500 0x7f9fa81514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa806def0 msgr2=0x7f9fa80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa806def0 0x7f9fa80b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fa8159500 msgr2=0x7f9fa81514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fa8159500 0x7f9fa81514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 -- 172.21.15.7:0/2112872827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f90009420 con 0x7f9fa80a9900 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fa77fe700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8159dd0 tx=0x7f9f90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011b10 con 0x7f9fa80a9900 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90000c90 con 0x7f9fa80a9900 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9001a640 con 0x7f9fa80a9900 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa815c690 con 0x7f9fa80a9900 2023-10-06T11:22:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa81519e0 con 0x7f9fa80a9900 2023-10-06T11:22:00.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa815c210 con 0x7f9fa80a9900 2023-10-06T11:22:00.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.743+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9f9001aa30 con 0x7f9fa80a9900 2023-10-06T11:22:00.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.747+0000 7f9f9ffff700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 0x7f9f88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.747+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f9f9007e6c0 con 0x7f9fa80a9900 2023-10-06T11:22:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.747+0000 7f9fa7fff700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 0x7f9f88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.747+0000 7f9fa7fff700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 0x7f9f88062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9f98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.755+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9fa815c210 con 0x7f9fa80a9900 2023-10-06T11:22:00.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:00.919+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9fa815c210 con 0x7f9fa80a9900 2023-10-06T11:22:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.603+0000 7f9f9ffff700 1 -- 172.21.15.7:0/2112872827 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9fa815c210 con 0x7f9fa80a9900 2023-10-06T11:22:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 msgr2=0x7f9f88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 0x7f9f88062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9f98009040 comp rx=0 tx=0).stop 2023-10-06T11:22:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 msgr2=0x7f9fa8153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8159dd0 tx=0x7f9f90017000 comp rx=0 tx=0).stop 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 shutdown_connections 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f88060550 0x7f9f88062a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fa8159500 0x7f9fa81514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80a9900 0x7f9fa8153c00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 --2- 172.21.15.7:0/2112872827 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9fa806def0 0x7f9fa80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 >> 172.21.15.7:0/2112872827 conn(0x7f9fa8068900 msgr2=0x7f9fa806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 shutdown_connections 2023-10-06T11:22:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.639+0000 7f9fae67e700 1 -- 172.21.15.7:0/2112872827 wait complete. 2023-10-06T11:22:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 5 allow_ec_overwrites to true 2023-10-06T11:22:01.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-06T11:22:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 Processor -- start 2023-10-06T11:22:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 -- start start 2023-10-06T11:22:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 0x7f71ac15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac15f320 con 0x7f71ac0af060 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0aecd0 con 0x7f71ac15e170 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b26a2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac15e710 con 0x7f71ac15ce30 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b0e9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b0e9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55182/0 (socket says 172.21.15.7:55182) 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b0e9f700 1 -- 172.21.15.7:0/3888246903 learned_addr learned my addr 172.21.15.7:0/3888246903 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 --2- 172.21.15.7:0/3888246903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 0x7f71ac15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 -- 172.21.15.7:0/3888246903 >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 msgr2=0x7f71ac0ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 --2- 172.21.15.7:0/3888246903 >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 -- 172.21.15.7:0/3888246903 >> v1:172.21.15.7:6789/0 conn(0x7f71ac0af060 legacy=0x7f71ac160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 -- 172.21.15.7:0/3888246903 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac15e9f0 con 0x7f71ac15ce30 2023-10-06T11:22:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b0e9f700 1 --2- 172.21.15.7:0/3888246903 >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71b16a0700 1 --2- 172.21.15.7:0/3888246903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 0x7f71ac15d200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e60 tx=0x7f71a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3b9ea8a1702bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71a3fff700 1 -- 172.21.15.7:0/3888246903 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400e070 con 0x7f71ac15ce30 2023-10-06T11:22:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71a3fff700 1 -- 172.21.15.7:0/3888246903 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a400bb60 con 0x7f71ac15ce30 2023-10-06T11:22:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.755+0000 7f71a3fff700 1 -- 172.21.15.7:0/3888246903 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012430 con 0x7f71ac15ce30 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 -- 172.21.15.7:0/3888246903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 msgr2=0x7f71ac15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 --2- 172.21.15.7:0/3888246903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 0x7f71ac15d200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e60 tx=0x7f71a4009780 comp rx=0 tx=0).stop 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 -- 172.21.15.7:0/3888246903 shutdown_connections 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 --2- 172.21.15.7:0/3888246903 >> v2:172.21.15.7:3300/0 conn(0x7f71ac15e170 0x7f71ac0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 --2- 172.21.15.7:0/3888246903 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15ce30 0x7f71ac15d200 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 -- 172.21.15.7:0/3888246903 >> 172.21.15.7:0/3888246903 conn(0x7f71ac0cc3c0 msgr2=0x7f71ac067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 -- 172.21.15.7:0/3888246903 shutdown_connections 2023-10-06T11:22:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 -- 172.21.15.7:0/3888246903 wait complete. 2023-10-06T11:22:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.759+0000 7f71b26a2700 1 Processor -- start 2023-10-06T11:22:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- start start 2023-10-06T11:22:01.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71ac0af060 0x7f71ac153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15e170 0x7f71ac1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac156440 con 0x7f71ac15ce30 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac154370 con 0x7f71ac0af060 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac155550 con 0x7f71ac15e170 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55192/0 (socket says 172.21.15.7:55192) 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 -- 172.21.15.7:0/4064614939 learned_addr learned my addr 172.21.15.7:0/4064614939 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b16a0700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71ac0af060 0x7f71ac153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b1ea1700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15e170 0x7f71ac1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 -- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15e170 msgr2=0x7f71ac1568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15e170 0x7f71ac1568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 -- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71ac0af060 msgr2=0x7f71ac153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71ac0af060 0x7f71ac153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 -- 172.21.15.7:0/4064614939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0094f0 con 0x7f71ac15ce30 2023-10-06T11:22:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b0e9f700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f719c00ec50 tx=0x7f719c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00c850 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00c9b0 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a4009a20 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00cd30 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.763+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac155b10 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.767+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac06c380 con 0x7f71ac15ce30 2023-10-06T11:22:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.767+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f719c013680 con 0x7f71ac15ce30 2023-10-06T11:22:01.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.767+0000 7f71a17fa700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 0x7f7190062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:01.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.767+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f719c07d5b0 con 0x7f71ac15ce30 2023-10-06T11:22:01.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.767+0000 7f71b16a0700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 0x7f7190062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:01.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.771+0000 7f71b16a0700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 0x7f7190062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097c0 tx=0x7f71a4019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.775+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f719c04d6d0 con 0x7f71ac15ce30 2023-10-06T11:22:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:01.975+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f71ac15ed30 con 0x7f71ac15ce30 2023-10-06T11:22:02.291 INFO:tasks.ceph.mds.a.smithi007.stderr:2023-10-06T11:22:02.287+0000 7f86cfcc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:22:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.631+0000 7f71a17fa700 1 -- 172.21.15.7:0/4064614939 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f719c050cf0 con 0x7f71ac15ce30 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 msgr2=0x7f7190062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 0x7f7190062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097c0 tx=0x7f71a4019000 comp rx=0 tx=0).stop 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 msgr2=0x7f71ac0a7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f719c00ec50 tx=0x7f719c009250 comp rx=0 tx=0).stop 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 shutdown_connections 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7190060550 0x7f7190062a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f71ac15e170 0x7f71ac1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac15ce30 0x7f71ac0a7df0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 --2- 172.21.15.7:0/4064614939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71ac0af060 0x7f71ac153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 >> 172.21.15.7:0/4064614939 conn(0x7f71ac0cc3c0 msgr2=0x7f71ac06d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 shutdown_connections 2023-10-06T11:22:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.635+0000 7f71b26a2700 1 -- 172.21.15.7:0/4064614939 wait complete. 2023-10-06T11:22:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 5 to fsmap 2023-10-06T11:22:02.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 Processor -- start 2023-10-06T11:22:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 -- start start 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d54157060 0x7f6d54156340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d541574e0 0x7f6d54156880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d54155eb0 con 0x7f6d54155630 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.759+0000 7f6d5b975700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d54156dc0 con 0x7f6d54157060 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5406a870 con 0x7f6d541574e0 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d54157060 0x7f6d54156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d54157060 0x7f6d54156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37694/0 (socket says 172.21.15.7:37694) 2023-10-06T11:22:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 -- 172.21.15.7:0/1753712513 learned_addr learned my addr 172.21.15.7:0/1753712513 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d59f12700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d541574e0 0x7f6d54156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 -- 172.21.15.7:0/1753712513 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d541574e0 msgr2=0x7f6d54156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d541574e0 0x7f6d54156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 -- 172.21.15.7:0/1753712513 >> v1:172.21.15.50:6789/0 conn(0x7f6d54155630 legacy=0x7f6d5406a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 -- 172.21.15.7:0/1753712513 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5406ab50 con 0x7f6d54157060 2023-10-06T11:22:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d58f10700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d54157060 0x7f6d54156340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d440099b0 tx=0x7f6d44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710ead8fb1105631 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d4bfff700 1 -- 172.21.15.7:0/1753712513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400b860 con 0x7f6d54157060 2023-10-06T11:22:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d4bfff700 1 -- 172.21.15.7:0/1753712513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4400b9c0 con 0x7f6d54157060 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d4bfff700 1 -- 172.21.15.7:0/1753712513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400bd60 con 0x7f6d54157060 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- 172.21.15.7:0/1753712513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d54157060 msgr2=0x7f6d54156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d54157060 0x7f6d54156340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d440099b0 tx=0x7f6d44009250 comp rx=0 tx=0).stop 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- 172.21.15.7:0/1753712513 shutdown_connections 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d541574e0 0x7f6d54156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 --2- 172.21.15.7:0/1753712513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d54157060 0x7f6d54156340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- 172.21.15.7:0/1753712513 >> 172.21.15.7:0/1753712513 conn(0x7f6d54067e30 msgr2=0x7f6d54068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- 172.21.15.7:0/1753712513 shutdown_connections 2023-10-06T11:22:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.763+0000 7f6d5b975700 1 -- 172.21.15.7:0/1753712513 wait complete. 2023-10-06T11:22:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 Processor -- start 2023-10-06T11:22:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 -- start start 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d54155630 0x7f6d541681d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d54157060 0x7f6d54150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540c1720 con 0x7f6d541574e0 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54168710 con 0x7f6d54155630 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d541513a0 con 0x7f6d54157060 2023-10-06T11:22:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37698/0 (socket says 172.21.15.7:37698) 2023-10-06T11:22:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 -- 172.21.15.7:0/2386214799 learned_addr learned my addr 172.21.15.7:0/2386214799 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59711700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d54155630 0x7f6d541681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d58f10700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d54157060 0x7f6d54150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 -- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d54157060 msgr2=0x7f6d54150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d54157060 0x7f6d54150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 -- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d54155630 msgr2=0x7f6d541681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d54155630 0x7f6d541681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 -- 172.21.15.7:0/2386214799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440094f0 con 0x7f6d541574e0 2023-10-06T11:22:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d59f12700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007f30 tx=0x7f6d4c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00f040 con 0x7f6d541574e0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c00d960 con 0x7f6d541574e0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00dd20 con 0x7f6d541574e0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.767+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d540c2930 con 0x7f6d541574e0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d540c2ef0 con 0x7f6d541574e0 2023-10-06T11:22:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5404be00 con 0x7f6d541574e0 2023-10-06T11:22:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6d4c008450 con 0x7f6d541574e0 2023-10-06T11:22:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d497fa700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 0x7f6d38062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6d4c07cd20 con 0x7f6d541574e0 2023-10-06T11:22:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.771+0000 7f6d59711700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 0x7f6d38062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.775+0000 7f6d59711700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 0x7f6d38062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d5415a4f0 tx=0x7f6d50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.779+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6d4c04cec0 con 0x7f6d541574e0 2023-10-06T11:22:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.971+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d54068a40 con 0x7f6d541574e0 2023-10-06T11:22:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d497fa700 1 -- 172.21.15.7:0/2386214799 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f6d4c0504e0 con 0x7f6d541574e0 2023-10-06T11:22:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:02.979 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:02.617028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 msgr2=0x7f6d38062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 0x7f6d38062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d5415a4f0 tx=0x7f6d50009040 comp rx=0 tx=0).stop 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 msgr2=0x7f6d54157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007f30 tx=0x7f6d4c00c040 comp rx=0 tx=0).stop 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 shutdown_connections 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d38060550 0x7f6d38062a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d541574e0 0x7f6d54157960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d54157060 0x7f6d54150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 --2- 172.21.15.7:0/2386214799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d54155630 0x7f6d541681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 >> 172.21.15.7:0/2386214799 conn(0x7f6d54067e30 msgr2=0x7f6d5406e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 shutdown_connections 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:02.975+0000 7f6d5b975700 1 -- 172.21.15.7:0/2386214799 wait complete. 2023-10-06T11:22:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2023-10-06T11:22:02.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 Processor -- start 2023-10-06T11:22:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 -- start start 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53e00c8ae0 0x7f53e006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e006ca30 con 0x7f53e00c9e00 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e006be20 con 0x7f53e00cb530 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53dffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e0159350 con 0x7f53e00c8ae0 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53deffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53e00c8ae0 0x7f53e006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53d7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f53e00c9e00 0x7f53e00cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52000/0 (socket says 172.21.15.7:52000) 2023-10-06T11:22:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.095+0000 7f53d7fff700 1 -- 172.21.15.7:0/2457781410 learned_addr learned my addr 172.21.15.7:0/2457781410 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909464962 0 0) 0x7f53e006ca30 con 0x7f53e00c9e00 2023-10-06T11:22:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53cc003570 con 0x7f53e00c9e00 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804250603 0 0) 0x7f53e006be20 con 0x7f53e00cb530 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e006ca30 con 0x7f53e00cb530 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471294084 0 0) 0x7f53cc003570 con 0x7f53e00c9e00 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53deffd700 1 -- 172.21.15.7:0/2457781410 >> v1:172.21.15.50:6789/0 conn(0x7f53e00cb530 legacy=0x7f53e00c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53deffd700 1 -- 172.21.15.7:0/2457781410 >> v1:172.21.15.7:6789/0 conn(0x7f53e00c9e00 legacy=0x7f53e00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53deffd700 1 -- 172.21.15.7:0/2457781410 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e006c100 con 0x7f53e00c8ae0 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53deffd700 1 --2- 172.21.15.7:0/2457781410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00c8ae0 0x7f53e006cec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098e0 tx=0x7f53d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a113dc95d26ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8001470 con 0x7f53e00c8ae0 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0011420 con 0x7f53e00c8ae0 2023-10-06T11:22:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53de7fc700 1 -- 172.21.15.7:0/2457781410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00117c0 con 0x7f53e00c8ae0 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 -- 172.21.15.7:0/2457781410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00c8ae0 msgr2=0x7f53e006cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 --2- 172.21.15.7:0/2457781410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00c8ae0 0x7f53e006cec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098e0 tx=0x7f53d00091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 -- 172.21.15.7:0/2457781410 shutdown_connections 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 --2- 172.21.15.7:0/2457781410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00c8ae0 0x7f53e006cec0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 -- 172.21.15.7:0/2457781410 >> 172.21.15.7:0/2457781410 conn(0x7f53e006e890 msgr2=0x7f53e006ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 -- 172.21.15.7:0/2457781410 shutdown_connections 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.099+0000 7f53dffff700 1 -- 172.21.15.7:0/2457781410 wait complete. 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 Processor -- start 2023-10-06T11:22:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- start start 2023-10-06T11:22:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53e00c8ae0 0x7f53e0148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53e00c9e00 0x7f53e015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c47b0 con 0x7f53e00cb530 2023-10-06T11:22:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0156b90 con 0x7f53e00c8ae0 2023-10-06T11:22:03.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0155f80 con 0x7f53e00c9e00 2023-10-06T11:22:03.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37704/0 (socket says 172.21.15.7:37704) 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 -- 172.21.15.7:0/2358641230 learned_addr learned my addr 172.21.15.7:0/2358641230 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53deffd700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53e00c8ae0 0x7f53e0148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53d7fff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53e00c9e00 0x7f53e015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 -- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53e00c9e00 msgr2=0x7f53e015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53e00c9e00 0x7f53e015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 -- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53e00c8ae0 msgr2=0x7f53e0148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53e00c8ae0 0x7f53e0148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 -- 172.21.15.7:0/2358641230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009420 con 0x7f53e00cb530 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53df7fe700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c860 tx=0x7f53d8002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8002030 con 0x7f53e00cb530 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d80021b0 con 0x7f53e00cb530 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0156260 con 0x7f53e00cb530 2023-10-06T11:22:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012630 con 0x7f53e00cb530 2023-10-06T11:22:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.103+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0156820 con 0x7f53e00cb530 2023-10-06T11:22:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e006d7f0 con 0x7f53e00cb530 2023-10-06T11:22:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f53d8012a20 con 0x7f53e00cb530 2023-10-06T11:22:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53d6ffd700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 0x7f53c0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f53d807cc10 con 0x7f53e00cb530 2023-10-06T11:22:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53deffd700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 0x7f53c0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.107+0000 7f53deffd700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 0x7f53c0062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53d00093f0 tx=0x7f53d0016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.115+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f53d804cd10 con 0x7f53e00cb530 2023-10-06T11:22:03.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.279+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53e0156cd0 con 0x7f53e00cb530 2023-10-06T11:22:03.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.279+0000 7f53d6ffd700 1 -- 172.21.15.7:0/2358641230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f53d8050330 con 0x7f53e00cb530 2023-10-06T11:22:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:03.287 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 msgr2=0x7f53c0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 0x7f53c0062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53d00093f0 tx=0x7f53d0016000 comp rx=0 tx=0).stop 2023-10-06T11:22:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 msgr2=0x7f53e015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c860 tx=0x7f53d8002700 comp rx=0 tx=0).stop 2023-10-06T11:22:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 shutdown_connections 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f53c0060550 0x7f53c0062a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53e00cb530 0x7f53e015adf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53e00c9e00 0x7f53e015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 --2- 172.21.15.7:0/2358641230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53e00c8ae0 0x7f53e0148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 >> 172.21.15.7:0/2358641230 conn(0x7f53e006e890 msgr2=0x7f53e0158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 shutdown_connections 2023-10-06T11:22:03.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.283+0000 7f53dffff700 1 -- 172.21.15.7:0/2358641230 wait complete. 2023-10-06T11:22:03.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.403+0000 7f4d19039700 1 Processor -- start 2023-10-06T11:22:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- start start 2023-10-06T11:22:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4d14068540 0x7f4d1415b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d14066840 0x7f4d140aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d140aea40 con 0x7f4d1415ac60 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d14068cb0 con 0x7f4d14066840 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140ae1e0 con 0x7f4d14068540 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d14066840 0x7f4d140aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d14066840 0x7f4d140aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37716/0 (socket says 172.21.15.7:37716) 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 -- 172.21.15.7:0/2224579813 learned_addr learned my addr 172.21.15.7:0/2224579813 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d0bfff700 1 --2- 172.21.15.7:0/2224579813 >> v2:172.21.15.50:3300/0 conn(0x7f4d14068540 0x7f4d1415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 -- 172.21.15.7:0/2224579813 >> v2:172.21.15.50:3300/0 conn(0x7f4d14068540 msgr2=0x7f4d1415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 --2- 172.21.15.7:0/2224579813 >> v2:172.21.15.50:3300/0 conn(0x7f4d14068540 0x7f4d1415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 -- 172.21.15.7:0/2224579813 >> v1:172.21.15.50:6789/0 conn(0x7f4d1415ac60 legacy=0x7f4d1406eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 -- 172.21.15.7:0/2224579813 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140689d0 con 0x7f4d14066840 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d18838700 1 --2- 172.21.15.7:0/2224579813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14066840 0x7f4d140aef20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d1415bb00 tx=0x7f4d0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e366cb2f5644eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d137fe700 1 -- 172.21.15.7:0/2224579813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00e070 con 0x7f4d14066840 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d137fe700 1 -- 172.21.15.7:0/2224579813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00bab0 con 0x7f4d14066840 2023-10-06T11:22:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d137fe700 1 -- 172.21.15.7:0/2224579813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0092e0 con 0x7f4d14066840 2023-10-06T11:22:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- 172.21.15.7:0/2224579813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14066840 msgr2=0x7f4d140aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 --2- 172.21.15.7:0/2224579813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14066840 0x7f4d140aef20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d1415bb00 tx=0x7f4d0c009840 comp rx=0 tx=0).stop 2023-10-06T11:22:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- 172.21.15.7:0/2224579813 shutdown_connections 2023-10-06T11:22:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 --2- 172.21.15.7:0/2224579813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14066840 0x7f4d140aef20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 --2- 172.21.15.7:0/2224579813 >> v2:172.21.15.50:3300/0 conn(0x7f4d14068540 0x7f4d1415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- 172.21.15.7:0/2224579813 >> 172.21.15.7:0/2224579813 conn(0x7f4d1406a990 msgr2=0x7f4d14067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- 172.21.15.7:0/2224579813 shutdown_connections 2023-10-06T11:22:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.407+0000 7f4d19039700 1 -- 172.21.15.7:0/2224579813 wait complete. 2023-10-06T11:22:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 Processor -- start 2023-10-06T11:22:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 -- start start 2023-10-06T11:22:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d14066840 0x7f4d140c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4d1415ac60 0x7f4d140c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c63a0 con 0x7f4d14068540 2023-10-06T11:22:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c4d50 con 0x7f4d14066840 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d19039700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c5030 con 0x7f4d1415ac60 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d0bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d0bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37722/0 (socket says 172.21.15.7:37722) 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d0bfff700 1 -- 172.21.15.7:0/3949697289 learned_addr learned my addr 172.21.15.7:0/3949697289 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d13fff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d14066840 0x7f4d140c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.411+0000 7f4d18838700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4d1415ac60 0x7f4d140c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 -- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4d1415ac60 msgr2=0x7f4d140c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4d1415ac60 0x7f4d140c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 -- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d14066840 msgr2=0x7f4d140c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d14066840 0x7f4d140c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 -- 172.21.15.7:0/3949697289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c009ae0 con 0x7f4d14068540 2023-10-06T11:22:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d0bfff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00ec80 tx=0x7f4cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00c850 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00c9b0 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d140c5310 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00cd50 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d140b1020 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d140c54a0 con 0x7f4d14068540 2023-10-06T11:22:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4cfc013680 con 0x7f4d14068540 2023-10-06T11:22:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 0x7f4cf4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.415+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4cfc07cf20 con 0x7f4d14068540 2023-10-06T11:22:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.419+0000 7f4d13fff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 0x7f4cf4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.419+0000 7f4d13fff700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 0x7f4cf4062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4d040039e0 tx=0x7f4d04007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.427+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4cfc04d020 con 0x7f4d14068540 2023-10-06T11:22:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.583+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d1406b8a0 con 0x7f4d14068540 2023-10-06T11:22:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.583+0000 7f4d10ff9700 1 -- 172.21.15.7:0/3949697289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f4cfc050640 con 0x7f4d14068540 2023-10-06T11:22:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:03.591 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 msgr2=0x7f4cf4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 0x7f4cf4062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4d040039e0 tx=0x7f4d04007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 msgr2=0x7f4d14150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00ec80 tx=0x7f4cfc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 shutdown_connections 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4cf4060550 0x7f4cf4062a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4d1415ac60 0x7f4d140c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d14068540 0x7f4d14150940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 --2- 172.21.15.7:0/3949697289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d14066840 0x7f4d140c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 >> 172.21.15.7:0/3949697289 conn(0x7f4d1406a990 msgr2=0x7f4d141521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 shutdown_connections 2023-10-06T11:22:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.587+0000 7f4d19039700 1 -- 172.21.15.7:0/3949697289 wait complete. 2023-10-06T11:22:03.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.723+0000 7f1100fa1700 1 Processor -- start 2023-10-06T11:22:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.723+0000 7f1100fa1700 1 -- start start 2023-10-06T11:22:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10fc06be20 0x7f10fc06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06d2f0 0x7f10fc15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0c80a0 con 0x7f10fc0ad990 2023-10-06T11:22:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc15d1d0 con 0x7f10fc06be20 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc15d470 con 0x7f10fc06d2f0 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10fc06be20 0x7f10fc06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f10fc0ad990 0x7f10fc1589a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52014/0 (socket says 172.21.15.7:52014) 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fbfff700 1 -- 172.21.15.7:0/1696217835 learned_addr learned my addr 172.21.15.7:0/1696217835 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/1696217835 >> v2:172.21.15.7:3300/0 conn(0x7f10fc06be20 0x7f10fc06c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37728/0 (socket says 172.21.15.7:37728) 2023-10-06T11:22:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10faffd700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06d2f0 0x7f10fc15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 -- 172.21.15.7:0/1696217835 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06d2f0 msgr2=0x7f10fc15cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06d2f0 0x7f10fc15cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 -- 172.21.15.7:0/1696217835 >> v1:172.21.15.7:6789/0 conn(0x7f10fc0ad990 legacy=0x7f10fc1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 -- 172.21.15.7:0/1696217835 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc15d750 con 0x7f10fc06be20 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06be20 0x7f10fc06c1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10f00098e0 tx=0x7f10f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376c0d90502d996b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fa7fc700 1 -- 172.21.15.7:0/1696217835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000b640 con 0x7f10fc06be20 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fa7fc700 1 -- 172.21.15.7:0/1696217835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f000b7a0 con 0x7f10fc06be20 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f10fa7fc700 1 -- 172.21.15.7:0/1696217835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000bb40 con 0x7f10fc06be20 2023-10-06T11:22:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- 172.21.15.7:0/1696217835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06be20 msgr2=0x7f10fc06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06be20 0x7f10fc06c1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10f00098e0 tx=0x7f10f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- 172.21.15.7:0/1696217835 shutdown_connections 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06d2f0 0x7f10fc15cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 --2- 172.21.15.7:0/1696217835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06be20 0x7f10fc06c1f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- 172.21.15.7:0/1696217835 >> 172.21.15.7:0/1696217835 conn(0x7f10fc0cdb00 msgr2=0x7f10fc0cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- 172.21.15.7:0/1696217835 shutdown_connections 2023-10-06T11:22:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.727+0000 7f1100fa1700 1 -- 172.21.15.7:0/1696217835 wait complete. 2023-10-06T11:22:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 Processor -- start 2023-10-06T11:22:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 -- start start 2023-10-06T11:22:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06be20 0x7f10fc147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 0x7f10fc0a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0ab690 con 0x7f10fc06d2f0 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0aada0 con 0x7f10fc0ad990 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f1100fa1700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0ab040 con 0x7f10fc06be20 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10faffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10faffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37736/0 (socket says 172.21.15.7:37736) 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10faffd700 1 -- 172.21.15.7:0/990678755 learned_addr learned my addr 172.21.15.7:0/990678755 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 0x7f10fc0a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06be20 0x7f10fc147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 -- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06be20 msgr2=0x7f10fc147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06be20 0x7f10fc147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 -- 172.21.15.7:0/990678755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 msgr2=0x7f10fc0a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 -- 172.21.15.7:0/990678755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009420 con 0x7f10fc0ad990 2023-10-06T11:22:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10faffd700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.731+0000 7f10fbfff700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 0x7f10fc0a8ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10f400cb40 tx=0x7f10f4001900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400a7e0 con 0x7f10fc0ad990 2023-10-06T11:22:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f400a940 con 0x7f10fc0ad990 2023-10-06T11:22:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400ace0 con 0x7f10fc0ad990 2023-10-06T11:22:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc0ab320 con 0x7f10fc0ad990 2023-10-06T11:22:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc0b32b0 con 0x7f10fc0ad990 2023-10-06T11:22:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.735+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc0ab800 con 0x7f10fc0ad990 2023-10-06T11:22:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.739+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f10f4035b50 con 0x7f10fc0ad990 2023-10-06T11:22:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.739+0000 7f10e37fe700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 0x7f10e8062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.739+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f10f407d060 con 0x7f10fc0ad990 2023-10-06T11:22:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.739+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 0x7f10e8062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.739+0000 7f10fb7fe700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 0x7f10e8062ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10f00093f0 tx=0x7f10f0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.747+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f10f404d1d0 con 0x7f10fc0ad990 2023-10-06T11:22:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10fc15da30 con 0x7f10fc0ad990 2023-10-06T11:22:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f10e37fe700 1 -- 172.21.15.7:0/990678755 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f10f4054e60 con 0x7f10fc0ad990 2023-10-06T11:22:03.952 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:03.952 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:02.617028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T11:22:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 msgr2=0x7f10e8062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 0x7f10e8062ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10f00093f0 tx=0x7f10f0017000 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 msgr2=0x7f10fc0a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 0x7f10fc0a8ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10f400cb40 tx=0x7f10f4001900 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 shutdown_connections 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f10e8060620 0x7f10e8062ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10fc0ad990 0x7f10fc0a8ee0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10fc06d2f0 0x7f10fc0a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 --2- 172.21.15.7:0/990678755 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f10fc06be20 0x7f10fc147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 >> 172.21.15.7:0/990678755 conn(0x7f10fc0cdb00 msgr2=0x7f10fc159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.947+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 shutdown_connections 2023-10-06T11:22:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:03.951+0000 7f1100fa1700 1 -- 172.21.15.7:0/990678755 wait complete. 2023-10-06T11:22:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2023-10-06T11:22:03.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.067+0000 7f6917fff700 1 Processor -- start 2023-10-06T11:22:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.067+0000 7f6917fff700 1 -- start start 2023-10-06T11:22:04.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 0x7f691806bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691806e6f0 con 0x7f6918155850 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69180cbc70 con 0x7f691806bad0 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180cbf50 con 0x7f691806eb80 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69177fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6918155850 0x7f6918155cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52030/0 (socket says 172.21.15.7:52030) 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69177fe700 1 -- 172.21.15.7:0/944107186 learned_addr learned my addr 172.21.15.7:0/944107186 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69167fc700 1 --2- 172.21.15.7:0/944107186 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 0x7f691806bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847471214 0 0) 0x7f691806e6f0 con 0x7f6918155850 2023-10-06T11:22:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6904003570 con 0x7f6918155850 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916206166 0 0) 0x7f69180cbc70 con 0x7f691806bad0 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691806e6f0 con 0x7f691806bad0 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69167fc700 1 -- 172.21.15.7:0/944107186 >> v1:172.21.15.50:6789/0 conn(0x7f691806bad0 legacy=0x7f69180cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69167fc700 1 -- 172.21.15.7:0/944107186 >> v1:172.21.15.7:6789/0 conn(0x7f6918155850 legacy=0x7f6918155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567530845 0 0) 0x7f6904003570 con 0x7f6918155850 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69167fc700 1 -- 172.21.15.7:0/944107186 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691806e0a0 con 0x7f691806eb80 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f69167fc700 1 --2- 172.21.15.7:0/944107186 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 0x7f691806bea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6900009910 tx=0x7f69000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e14adcfa6f160e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011630 con 0x7f691806eb80 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900011790 con 0x7f691806eb80 2023-10-06T11:22:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6915ffb700 1 -- 172.21.15.7:0/944107186 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011b30 con 0x7f691806eb80 2023-10-06T11:22:04.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- 172.21.15.7:0/944107186 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 msgr2=0x7f691806bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 --2- 172.21.15.7:0/944107186 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 0x7f691806bea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6900009910 tx=0x7f69000091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- 172.21.15.7:0/944107186 shutdown_connections 2023-10-06T11:22:04.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 --2- 172.21.15.7:0/944107186 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691806eb80 0x7f691806bea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- 172.21.15.7:0/944107186 >> 172.21.15.7:0/944107186 conn(0x7f69180cc190 msgr2=0x7f69180cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- 172.21.15.7:0/944107186 shutdown_connections 2023-10-06T11:22:04.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.071+0000 7f6917fff700 1 -- 172.21.15.7:0/944107186 wait complete. 2023-10-06T11:22:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.075+0000 7f6917fff700 1 Processor -- start 2023-10-06T11:22:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.075+0000 7f6917fff700 1 -- start start 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 0x7f6918158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6918155850 0x7f6918159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180c3240 con 0x7f691806bad0 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180c3490 con 0x7f691806eb80 2023-10-06T11:22:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180c3770 con 0x7f6918155850 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37740/0 (socket says 172.21.15.7:37740) 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 -- 172.21.15.7:0/2101365657 learned_addr learned my addr 172.21.15.7:0/2101365657 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f69167fc700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 0x7f6918158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f69177fe700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6918155850 0x7f6918159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 -- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6918155850 msgr2=0x7f6918159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6918155850 0x7f6918159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 -- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 msgr2=0x7f6918158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 0x7f6918158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 -- 172.21.15.7:0/2101365657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900009420 con 0x7f691806bad0 2023-10-06T11:22:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f69167fc700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 0x7f6918158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6916ffd700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f690800c790 tx=0x7f6908007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908010510 con 0x7f691806bad0 2023-10-06T11:22:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908010670 con 0x7f691806bad0 2023-10-06T11:22:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908010a10 con 0x7f691806bad0 2023-10-06T11:22:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69180c3a50 con 0x7f691806bad0 2023-10-06T11:22:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69180c4010 con 0x7f691806bad0 2023-10-06T11:22:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.079+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69180c4230 con 0x7f691806bad0 2023-10-06T11:22:04.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.083+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6908025430 con 0x7f691806bad0 2023-10-06T11:22:04.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.083+0000 7f690effd700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.083+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f690807d540 con 0x7f691806bad0 2023-10-06T11:22:04.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.083+0000 7f69167fc700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.083+0000 7f69167fc700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 0x7f68f8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6900009860 tx=0x7f6900017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.087+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f690804d6c0 con 0x7f691806bad0 2023-10-06T11:22:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.251+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f691806e380 con 0x7f691806bad0 2023-10-06T11:22:04.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f690effd700 1 -- 172.21.15.7:0/2101365657 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f6908050ce0 con 0x7f691806bad0 2023-10-06T11:22:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:04.260 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 msgr2=0x7f68f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 0x7f68f8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6900009860 tx=0x7f6900017000 comp rx=0 tx=0).stop 2023-10-06T11:22:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 msgr2=0x7f691813e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f690800c790 tx=0x7f6908007e10 comp rx=0 tx=0).stop 2023-10-06T11:22:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 shutdown_connections 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6918155850 0x7f6918159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691806eb80 0x7f6918158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 --2- 172.21.15.7:0/2101365657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691806bad0 0x7f691813e970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 >> 172.21.15.7:0/2101365657 conn(0x7f69180cc190 msgr2=0x7f6918150210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 shutdown_connections 2023-10-06T11:22:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.255+0000 7f6917fff700 1 -- 172.21.15.7:0/2101365657 wait complete. 2023-10-06T11:22:04.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:21:56.923085+0000', 'modified': '2023-10-06T11:22:02.617028+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/932941616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 932941616}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 932941616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-06T11:22:04.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:22:04.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:22:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.791+0000 7fc73ceeb700 1 Processor -- start 2023-10-06T11:22:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.791+0000 7fc73ceeb700 1 -- start start 2023-10-06T11:22:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.791+0000 7fc73ceeb700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc7380ccc80 0x7fc7381563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.791+0000 7fc73ceeb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7381568e0 0x7fc73806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.791+0000 7fc73ceeb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73806d7b0 con 0x7fc7380cc8b0 2023-10-06T11:22:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7380cd100 con 0x7fc7381568e0 2023-10-06T11:22:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc738156d80 con 0x7fc7380ccc80 2023-10-06T11:22:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7381568e0 0x7fc73806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7381568e0 0x7fc73806dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37756/0 (socket says 172.21.15.7:37756) 2023-10-06T11:22:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 -- 172.21.15.7:0/1870776863 learned_addr learned my addr 172.21.15.7:0/1870776863 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc736ffd700 1 --2- 172.21.15.7:0/1870776863 >> v2:172.21.15.50:3300/0 conn(0x7fc7380ccc80 0x7fc7381563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 -- 172.21.15.7:0/1870776863 >> v2:172.21.15.50:3300/0 conn(0x7fc7380ccc80 msgr2=0x7fc7381563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 --2- 172.21.15.7:0/1870776863 >> v2:172.21.15.50:3300/0 conn(0x7fc7380ccc80 0x7fc7381563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 -- 172.21.15.7:0/1870776863 >> v1:172.21.15.7:6789/0 conn(0x7fc7380cc8b0 legacy=0x7fc738069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 -- 172.21.15.7:0/1870776863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73815a4d0 con 0x7fc7381568e0 2023-10-06T11:22:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc737fff700 1 --2- 172.21.15.7:0/1870776863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc73806dc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc73806d720 tx=0x7fc730009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea37d5a8467c7cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc7367fc700 1 -- 172.21.15.7:0/1870776863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730009e70 con 0x7fc7381568e0 2023-10-06T11:22:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc7367fc700 1 -- 172.21.15.7:0/1870776863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000b860 con 0x7fc7381568e0 2023-10-06T11:22:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc7367fc700 1 -- 172.21.15.7:0/1870776863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000bc20 con 0x7fc7381568e0 2023-10-06T11:22:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- 172.21.15.7:0/1870776863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 msgr2=0x7fc73806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/1870776863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc73806dc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc73806d720 tx=0x7fc730009250 comp rx=0 tx=0).stop 2023-10-06T11:22:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- 172.21.15.7:0/1870776863 shutdown_connections 2023-10-06T11:22:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/1870776863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc73806dc40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/1870776863 >> v2:172.21.15.50:3300/0 conn(0x7fc7380ccc80 0x7fc7381563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- 172.21.15.7:0/1870776863 >> 172.21.15.7:0/1870776863 conn(0x7fc738067e30 msgr2=0x7fc738068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- 172.21.15.7:0/1870776863 shutdown_connections 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.795+0000 7fc73ceeb700 1 -- 172.21.15.7:0/1870776863 wait complete. 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 Processor -- start 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- start start 2023-10-06T11:22:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc7380cc8b0 0x7fc7380cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7380ccc80 0x7fc7380cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381697b0 con 0x7fc7381568e0 2023-10-06T11:22:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380ce070 con 0x7fc7380ccc80 2023-10-06T11:22:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380ce350 con 0x7fc7380cc8b0 2023-10-06T11:22:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37772/0 (socket says 172.21.15.7:37772) 2023-10-06T11:22:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 -- 172.21.15.7:0/3744468515 learned_addr learned my addr 172.21.15.7:0/3744468515 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc736ffd700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7380ccc80 0x7fc7380cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc7380cc8b0 0x7fc7380cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 -- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc7380cc8b0 msgr2=0x7fc7380cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc7380cc8b0 0x7fc7380cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 -- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7380ccc80 msgr2=0x7fc7380cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7380ccc80 0x7fc7380cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 -- 172.21.15.7:0/3744468515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300094f0 con 0x7fc7381568e0 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc737fff700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc72f7fe700 1 -- 172.21.15.7:0/3744468515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73001f070 con 0x7fc7381568e0 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc72f7fe700 1 -- 172.21.15.7:0/3744468515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000b9d0 con 0x7fc7381568e0 2023-10-06T11:22:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7380ce630 con 0x7fc7381568e0 2023-10-06T11:22:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc72f7fe700 1 -- 172.21.15.7:0/3744468515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011420 con 0x7fc7381568e0 2023-10-06T11:22:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.799+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7380cdd80 con 0x7fc7381568e0 2023-10-06T11:22:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.803+0000 7fc72f7fe700 1 -- 172.21.15.7:0/3744468515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc730011790 con 0x7fc7381568e0 2023-10-06T11:22:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.803+0000 7fc72f7fe700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 0x7fc718062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.803+0000 7fc72f7fe700 1 -- 172.21.15.7:0/3744468515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc73007e080 con 0x7fc7381568e0 2023-10-06T11:22:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.803+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 0x7fc718062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.803+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 0x7fc718062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc738158e80 tx=0x7fc728009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 Processor -- start 2023-10-06T11:22:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 -- start start 2023-10-06T11:22:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ed9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1ed9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e07f30 con 0x1ca7790 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e03ed0 con 0x1c5cae0 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dca0b0 con 0x1c5c710 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37788/0 (socket says 172.21.15.7:37788) 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 -- 172.21.15.7:0/3000282299 learned_addr learned my addr 172.21.15.7:0/3000282299 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ed9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc736ffd700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1ed9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 -- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 msgr2=0x1ed9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1ed9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 -- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 msgr2=0x1ed9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ed9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 -- 172.21.15.7:0/3000282299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ca89e0 con 0x1ca7790 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ed9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc737fff700 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc73004e3c0 tx=0x7fc73007c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d371954e3e59a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.807+0000 7fc72cff9700 1 -- 172.21.15.7:0/3000282299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730055150 con 0x1ca7790 2023-10-06T11:22:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc72cff9700 1 -- 172.21.15.7:0/3000282299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730055e90 con 0x1ca7790 2023-10-06T11:22:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc72cff9700 1 -- 172.21.15.7:0/3000282299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730058ea0 con 0x1ca7790 2023-10-06T11:22:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- 172.21.15.7:0/3000282299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 msgr2=0x1eda0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc73004e3c0 tx=0x7fc73007c010 comp rx=0 tx=0).stop 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- 172.21.15.7:0/3000282299 shutdown_connections 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1eda0b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1ed9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3000282299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ed9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- 172.21.15.7:0/3000282299 >> 172.21.15.7:0/3000282299 conn(0x1c5ceb0 msgr2=0x1e96060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- 172.21.15.7:0/3000282299 shutdown_connections 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- 172.21.15.7:0/3000282299 wait complete. 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- start start 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 Processor -- start 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ede180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1e93c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dc9d50 con 0x1c5c710 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da2970 con 0x1c5cae0 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c33060 con 0x1ca7790 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7403f6740 10 client.0 _ll_register_callbacks cb 0x1da6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37802/0 (socket says 172.21.15.7:37802) 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 -- 172.21.15.7:0/3507861952 learned_addr learned my addr 172.21.15.7:0/3507861952 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc736ffd700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ede180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc737fff700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1e93c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 -- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 msgr2=0x1e93c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1e93c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 -- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 msgr2=0x1ede180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.811+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ede180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7377fe700 1 -- 172.21.15.7:0/3507861952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7380cdd80 con 0x1c5c710 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc737fff700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1e93c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7377fe700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc728014860 tx=0x7fc728015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c9ffe85658ab21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728021430 con 0x1c5c710 2023-10-06T11:22:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc72801c040 con 0x1c5c710 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1da2cc0 con 0x1c5c710 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7280217a0 con 0x1c5c710 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 10 client.4455 fetch_fsmap learned FSMap version 7 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x1dd2370 con 0x1c5c710 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1da2cc0 con 0x1c5c710 2023-10-06T11:22:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fc728021900 con 0x1c5c710 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 10 client.4455 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 10 client.4455 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 20 client.4455 populate_metadata read hostname 'smithi007' 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1e95280 0x1ee8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 4 client.4455 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 --> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1da63e0 con 0x1e95280 2023-10-06T11:22:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc736ffd700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1e95280 0x1ee8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 10 client.4455 ms_handle_connect on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc736ffd700 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1e95280 0x1ee8de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47554162bf1b9dbd server_cookie=2b27a259c18c3c2 in_seq=0 out_seq=0 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 1 -- 172.21.15.7:0/3507861952 <== mds.0 v2:172.21.15.7:6834/932941616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1da63e0 con 0x1e95280 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 10 client.4455 handle_command_reply: tid=0 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 client.4455 shutdown 2023-10-06T11:22:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 msgr2=0x1eddc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc728014860 tx=0x7fc728015000 comp rx=0 tx=0).stop 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 shutdown_connections 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1e95280 0x1ee8de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1e93c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1ede180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc715ffb700 0 client.4455 ms_handle_reset on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 --2- 172.21.15.7:0/3507861952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5c710 0x1eddc40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 >> 172.21.15.7:0/3507861952 conn(0x1c5ceb0 msgr2=0x1ebd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.815+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 shutdown_connections 2023-10-06T11:22:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.819+0000 7fc7403f6740 1 -- 172.21.15.7:0/3507861952 wait complete. 2023-10-06T11:22:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.819+0000 7fc7403f6740 20 client.4455 trim_cache size 0 max 16384 2023-10-06T11:22:04.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 Processor -- start 2023-10-06T11:22:04.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 -- start start 2023-10-06T11:22:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 0x1ebd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1edc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e02d60 con 0x1c5cae0 2023-10-06T11:22:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e06590 con 0x1c5c710 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cd57c0 con 0x1ca7790 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc736ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc736ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37806/0 (socket says 172.21.15.7:37806) 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.835+0000 7fc736ffd700 1 -- 172.21.15.7:0/4033368722 learned_addr learned my addr 172.21.15.7:0/4033368722 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7377fe700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 0x1ebd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc737fff700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1edc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 -- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 msgr2=0x1edc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1edc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 -- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 msgr2=0x1ebd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 0x1ebd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 -- 172.21.15.7:0/4033368722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7381582f0 con 0x1c5cae0 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7377fe700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 0x1ebd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc736ffd700 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc720000f80 tx=0x7fc72000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b836d639cf7c64ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc72d7fa700 1 -- 172.21.15.7:0/4033368722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000e040 con 0x1c5cae0 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc72d7fa700 1 -- 172.21.15.7:0/4033368722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000f040 con 0x1c5cae0 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc72d7fa700 1 -- 172.21.15.7:0/4033368722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200176f0 con 0x1c5cae0 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- 172.21.15.7:0/4033368722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 msgr2=0x1ebd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc720000f80 tx=0x7fc72000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- 172.21.15.7:0/4033368722 shutdown_connections 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1ca7790 0x1edc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1c5cae0 0x1ebd720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- 172.21.15.7:0/4033368722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5c710 0x1ebd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- 172.21.15.7:0/4033368722 >> 172.21.15.7:0/4033368722 conn(0x1c5ceb0 msgr2=0x1e8f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- 172.21.15.7:0/4033368722 shutdown_connections 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- 172.21.15.7:0/4033368722 wait complete. 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- start start 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 Processor -- start 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1f33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1f34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd9010 con 0x1ca7790 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd8610 con 0x1c5cae0 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e5a270 con 0x1c5c710 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.839+0000 7fc7403f6740 10 client.0 _ll_register_callbacks cb 0x1dcf110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:22:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37818/0 (socket says 172.21.15.7:37818) 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7377fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1f33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 -- 172.21.15.7:0/1494129177 learned_addr learned my addr 172.21.15.7:0/1494129177 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc736ffd700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1f34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 -- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 msgr2=0x1f33ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1f33ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 -- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 msgr2=0x1f34000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1f34000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 -- 172.21.15.7:0/1494129177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720007730 con 0x1ca7790 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc737fff700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc73000bd50 tx=0x7fc730053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4418117f7fea99cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-06T11:22:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730055150 con 0x1ca7790 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc730058ea0 con 0x1ca7790 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc730055e90 con 0x1ca7790 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc730067360 con 0x1ca7790 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 10 client.4461 fetch_fsmap learned FSMap version 7 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x1db8d80 con 0x1ca7790 2023-10-06T11:22:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730055e90 con 0x1ca7790 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fc730057070 con 0x1ca7790 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 10 client.4461 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 10 client.4461 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 20 client.4461 populate_metadata read hostname 'smithi007' 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1ee8730 0x1ee8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 4 client.4461 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 --> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1dd8410 con 0x1ee8730 2023-10-06T11:22:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7377fe700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1ee8730 0x1ee8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc715ffb700 10 client.4461 ms_handle_connect on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.843+0000 7fc7377fe700 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1ee8730 0x1ee8b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8cbb61b89265da server_cookie=2ea90e5f7f39b944 in_seq=0 out_seq=0 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc715ffb700 1 -- 172.21.15.7:0/1494129177 <== mds.0 v2:172.21.15.7:6834/932941616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x1dd8410 con 0x1ee8730 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc715ffb700 10 client.4461 handle_command_reply: tid=0 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 client.4461 shutdown 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 msgr2=0x1ee7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc73000bd50 tx=0x7fc730053000 comp rx=0 tx=0).stop 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 shutdown_connections 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x1ee8730 0x1ee8b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1ca7790 0x1ee7da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc715ffb700 0 client.4461 ms_handle_reset on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1c5cae0 0x1f34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 --2- 172.21.15.7:0/1494129177 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1c5c710 0x1f33ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 >> 172.21.15.7:0/1494129177 conn(0x1c5ceb0 msgr2=0x1ebd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 shutdown_connections 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 1 -- 172.21.15.7:0/1494129177 wait complete. 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc7403f6740 20 client.4461 trim_cache size 0 max 16384 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "88789558-6143-4e37-9ef9-c1260e585533", 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4287, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.2777053870000001, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 7, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 25, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 25, 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.592971768 2023-10-06T11:22:04.859 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 msgr2=0x7fc718062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 0x7fc718062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc738158e80 tx=0x7fc728009040 comp rx=0 tx=0).stop 2023-10-06T11:22:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 msgr2=0x7fc738152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730017000 comp rx=0 tx=0).stop 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 shutdown_connections 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc718060550 0x7fc718062a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7381568e0 0x7fc738152be0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7380ccc80 0x7fc7380cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 --2- 172.21.15.7:0/3744468515 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc7380cc8b0 0x7fc7380cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 >> 172.21.15.7:0/3744468515 conn(0x7fc738067e30 msgr2=0x7fc738153620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 shutdown_connections 2023-10-06T11:22:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.847+0000 7fc73ceeb700 1 -- 172.21.15.7:0/3744468515 wait complete. 2023-10-06T11:22:04.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:22:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.975+0000 7f7878d8f700 1 Processor -- start 2023-10-06T11:22:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.975+0000 7f7878d8f700 1 -- start start 2023-10-06T11:22:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f787406a3e0 0x7f787406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740abd70 con 0x7f78740c71e0 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787406c090 con 0x7f7874157970 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740c6dd0 con 0x7f787406a3e0 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7872ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78740c71e0 0x7f7874069cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52038/0 (socket says 172.21.15.7:52038) 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7872ffd700 1 -- 172.21.15.7:0/1772124493 learned_addr learned my addr 172.21.15.7:0/1772124493 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7873fff700 1 --2- 172.21.15.7:0/1772124493 >> v2:172.21.15.50:3300/0 conn(0x7f787406a3e0 0x7f787406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435826621 0 0) 0x7f78740abd70 con 0x7f78740c71e0 2023-10-06T11:22:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003570 con 0x7f78740c71e0 2023-10-06T11:22:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348810500 0 0) 0x7f787406c090 con 0x7f7874157970 2023-10-06T11:22:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78740abd70 con 0x7f7874157970 2023-10-06T11:22:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2664360482 0 0) 0x7f7860003570 con 0x7f78740c71e0 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7873fff700 1 -- 172.21.15.7:0/1772124493 >> v1:172.21.15.50:6789/0 conn(0x7f7874157970 legacy=0x7f7874157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7873fff700 1 -- 172.21.15.7:0/1772124493 >> v1:172.21.15.7:6789/0 conn(0x7f78740c71e0 legacy=0x7f7874069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7873fff700 1 -- 172.21.15.7:0/1772124493 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740c7670 con 0x7f787406a3e0 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7873fff700 1 --2- 172.21.15.7:0/1772124493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787406a3e0 0x7f787406e000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098e0 tx=0x7f786c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8506ee4d7ff4dee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c001470 con 0x7f787406a3e0 2023-10-06T11:22:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c011420 con 0x7f787406a3e0 2023-10-06T11:22:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f78727fc700 1 -- 172.21.15.7:0/1772124493 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0117c0 con 0x7f787406a3e0 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 -- 172.21.15.7:0/1772124493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787406a3e0 msgr2=0x7f787406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.979+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1772124493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787406a3e0 0x7f787406e000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098e0 tx=0x7f786c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 -- 172.21.15.7:0/1772124493 shutdown_connections 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1772124493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787406a3e0 0x7f787406e000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 -- 172.21.15.7:0/1772124493 >> 172.21.15.7:0/1772124493 conn(0x7f78740cc190 msgr2=0x7f78740cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 -- 172.21.15.7:0/1772124493 shutdown_connections 2023-10-06T11:22:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 -- 172.21.15.7:0/1772124493 wait complete. 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 Processor -- start 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 -- start start 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.983+0000 7f7878d8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 0x7f78740afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7874157970 0x7f7874166df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740aff40 con 0x7f787406a3e0 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b0130 con 0x7f78740c71e0 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b0410 con 0x7f7874157970 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7872ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 0x7f78740afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37822/0 (socket says 172.21.15.7:37822) 2023-10-06T11:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 -- 172.21.15.7:0/1985349488 learned_addr learned my addr 172.21.15.7:0/1985349488 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7873fff700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7874157970 0x7f7874166df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 -- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7874157970 msgr2=0x7f7874166df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7874157970 0x7f7874166df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 -- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 msgr2=0x7f78740afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 0x7f78740afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 -- 172.21.15.7:0/1985349488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009420 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7872ffd700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 0x7f78740afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f78737fe700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f786400cb10 tx=0x7f7864002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400ce20 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400a7e0 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740b06f0 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400ab80 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.987+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7874152ae0 con 0x7f787406a3e0 2023-10-06T11:22:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7874152d00 con 0x7f787406a3e0 2023-10-06T11:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7864012420 con 0x7f787406a3e0 2023-10-06T11:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f786b7fe700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 0x7f7854062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f786407cf10 con 0x7f787406a3e0 2023-10-06T11:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f7872ffd700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 0x7f7854062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:04.991+0000 7f7872ffd700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 0x7f7854062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f787406eb30 tx=0x7f785c001620 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.003+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f786404d0b0 con 0x7f787406a3e0 2023-10-06T11:22:05.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:05.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.199+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f787406ba40 con 0x7f787406a3e0 2023-10-06T11:22:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f786b7fe700 1 -- 172.21.15.7:0/1985349488 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f78640506d0 con 0x7f787406a3e0 2023-10-06T11:22:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T11:22:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 msgr2=0x7f7854062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 0x7f7854062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f787406eb30 tx=0x7f785c001620 comp rx=0 tx=0).stop 2023-10-06T11:22:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 msgr2=0x7f78740b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f786400cb10 tx=0x7f7864002820 comp rx=0 tx=0).stop 2023-10-06T11:22:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 shutdown_connections 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7854060550 0x7f7854062a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7874157970 0x7f7874166df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78740c71e0 0x7f78740afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 --2- 172.21.15.7:0/1985349488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f787406a3e0 0x7f78740b19e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 >> 172.21.15.7:0/1985349488 conn(0x7f78740cc190 msgr2=0x7f787406e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 shutdown_connections 2023-10-06T11:22:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:05.203+0000 7f7878d8f700 1 -- 172.21.15.7:0/1985349488 wait complete. 2023-10-06T11:22:05.221 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:22:05.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:22:05.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-06T11:22:05.222 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-06T11:22:05.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:05.222 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:22:05.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:22:05.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:22:05.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T11:22:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.223 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42577sec preferred_lft 42577sec 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:c9:9f:83:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:22:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:05.233 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:22:05.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:22:05.234 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:05.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.277 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:22:05.287 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-06T11:22:05.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.289 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-06T11:22:05.358 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-06T11:22:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:22:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:22:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:22:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:22:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:22:05.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.360 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:05.360 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:05.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:22:05.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:22:05.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:22:05.361 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:05.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.404 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:22:05.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.411 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-10-06T11:22:05.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.462 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:05.462 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:05.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T11:22:05.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-06T11:22:05.462 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:05.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-06T11:22:05.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-06T11:22:05.496 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:05.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:05.572 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:05.572 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:05.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-06T11:22:05.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:22:05.572 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:05.594 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-06T11:22:05.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:05.595 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:05.645 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T11:22:05.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:05.645 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:05.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:05.666 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:05.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:05.682 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:05.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:05.683 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:05.733 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:22:05.733 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-06T11:22:05.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:05.753 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:05.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:05.766 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:05.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:05.767 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:05.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2023-10-06T11:22:05.823+0000 7f91a4068380 -1 init, newargv = 0x55748054f610 newargc=15 2023-10-06T11:22:05.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[39298]: starting ceph client 2023-10-06T11:22:05.842 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[39298]: starting fuse 2023-10-06T11:22:06.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:06.935 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:06.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:06.947 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:06.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:06.947 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:06.952 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:22:06.952 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:06.953 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-06T11:22:06.954 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:22:06.955 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:06.955 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2023-10-06T11:22:06.955 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-06T11:22:07.076 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.39298.asok 2023-10-06T11:22:07.261 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.39298.asok 2023-10-06T11:22:07.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:07.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.39298.asok status 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2023-10-06T11:22:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "39298", 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4473, 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-06T11:22:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4473 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1362842004 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1362842004 2023-10-06T11:22:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4473 192.168.0.1:0/1362842004", 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1362842004", 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 7, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 25, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-06T11:22:07.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:07.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:07.363 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-06T11:22:07.363 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:07.371 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-06T11:22:07.371 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:07.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:07.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:07.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:07.431 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-06T11:22:07.474 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2023-10-06T11:22:07.474 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-06T11:22:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:07.475 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-06T11:22:07.476 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:22:07.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-06T11:22:07.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[39298]: fuse finished with error 0 and tester_r 0 2023-10-06T11:22:07.501 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:22:13.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:22:13.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:13.603 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-06T11:22:13.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 Processor -- start 2023-10-06T11:22:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 -- start start 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 0x7fa66c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c0b05c0 con 0x7fa66c0a88b0 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c15e790 con 0x7fa66c15d5a0 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa67402b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c15e2d0 con 0x7fa66c15d9c0 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa6725c8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa66c0a88b0 0x7fa66c0a8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54728/0 (socket says 172.21.15.7:54728) 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa6725c8700 1 -- 172.21.15.7:0/1643920045 learned_addr learned my addr 172.21.15.7:0/1643920045 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:13.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa671dc7700 1 --2- 172.21.15.7:0/1643920045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 0x7fa66c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:13.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835990298 0 0) 0x7fa66c0b05c0 con 0x7fa66c0a88b0 2023-10-06T11:22:13.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa658003570 con 0x7fa66c0a88b0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240897522 0 0) 0x7fa66c15e790 con 0x7fa66c15d5a0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa66c0b05c0 con 0x7fa66c15d5a0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa671dc7700 1 -- 172.21.15.7:0/1643920045 >> v1:172.21.15.50:6789/0 conn(0x7fa66c15d5a0 legacy=0x7fa66c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa671dc7700 1 -- 172.21.15.7:0/1643920045 >> v1:172.21.15.7:6789/0 conn(0x7fa66c0a88b0 legacy=0x7fa66c0a8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa671dc7700 1 -- 172.21.15.7:0/1643920045 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0af6d0 con 0x7fa66c15d9c0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa671dc7700 1 --2- 172.21.15.7:0/1643920045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 0x7fa66c15dd90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009900 tx=0x7fa65c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31235db54d64cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c005c10 con 0x7fa66c15d9c0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c005d70 con 0x7fa66c15d9c0 2023-10-06T11:22:13.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.775+0000 7fa670dc5700 1 -- 172.21.15.7:0/1643920045 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c006260 con 0x7fa66c15d9c0 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- 172.21.15.7:0/1643920045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 msgr2=0x7fa66c15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 --2- 172.21.15.7:0/1643920045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 0x7fa66c15dd90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009900 tx=0x7fa65c009120 comp rx=0 tx=0).stop 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- 172.21.15.7:0/1643920045 shutdown_connections 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 --2- 172.21.15.7:0/1643920045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d9c0 0x7fa66c15dd90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- 172.21.15.7:0/1643920045 >> 172.21.15.7:0/1643920045 conn(0x7fa66c0688c0 msgr2=0x7fa66c0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- 172.21.15.7:0/1643920045 shutdown_connections 2023-10-06T11:22:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- 172.21.15.7:0/1643920045 wait complete. 2023-10-06T11:22:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 Processor -- start 2023-10-06T11:22:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 -- start start 2023-10-06T11:22:13.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 0x7fa66c16d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:13.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d5a0 0x7fa66c157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:13.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.779+0000 7fa67402b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 0x7fa66c0adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0aae50 con 0x7fa66c15d9c0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0ab6f0 con 0x7fa66c0a88b0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c157760 con 0x7fa66c15d5a0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa671dc7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 0x7fa66c16d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 0x7fa66c0adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa671dc7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 0x7fa66c16d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57534/0 (socket says 172.21.15.7:57534) 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6715c6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d5a0 0x7fa66c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa671dc7700 1 -- 172.21.15.7:0/2047496770 learned_addr learned my addr 172.21.15.7:0/2047496770 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 -- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d5a0 msgr2=0x7fa66c157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d5a0 0x7fa66c157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 -- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 msgr2=0x7fa66c16d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 0x7fa66c16d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 -- 172.21.15.7:0/2047496770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0093c0 con 0x7fa66c15d9c0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa6725c8700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 0x7fa66c0adef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c50 tx=0x7fa664001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400a830 con 0x7fa66c15d9c0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66400a990 con 0x7fa66c15d9c0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0aea90 con 0x7fa66c15d9c0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400ad50 con 0x7fa66c15d9c0 2023-10-06T11:22:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c0ab380 con 0x7fa66c15d9c0 2023-10-06T11:22:13.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.783+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c15e500 con 0x7fa66c15d9c0 2023-10-06T11:22:13.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.787+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa664007300 con 0x7fa66c15d9c0 2023-10-06T11:22:13.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.787+0000 7fa661ffb700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 0x7fa650062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:13.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.787+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa66407cf10 con 0x7fa66c15d9c0 2023-10-06T11:22:13.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.787+0000 7fa671dc7700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 0x7fa650062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.787+0000 7fa671dc7700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 0x7fa650062a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009850 tx=0x7fa65c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:13.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.799+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa66c15e500 con 0x7fa66c15d9c0 2023-10-06T11:22:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.963+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa66c15e500 con 0x7fa66c15d9c0 2023-10-06T11:22:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa661ffb700 1 -- 172.21.15.7:0/2047496770 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7fa66c15e500 con 0x7fa66c15d9c0 2023-10-06T11:22:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:13.971 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 msgr2=0x7fa650062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 0x7fa650062a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009850 tx=0x7fa65c016000 comp rx=0 tx=0).stop 2023-10-06T11:22:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 msgr2=0x7fa66c0adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 0x7fa66c0adef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa664005c50 tx=0x7fa664001a40 comp rx=0 tx=0).stop 2023-10-06T11:22:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 shutdown_connections 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa650060550 0x7fa650062a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa66c15d9c0 0x7fa66c0adef0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa66c15d5a0 0x7fa66c157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 --2- 172.21.15.7:0/2047496770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa66c0a88b0 0x7fa66c16d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 >> 172.21.15.7:0/2047496770 conn(0x7fa66c0688c0 msgr2=0x7fa66c1618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 shutdown_connections 2023-10-06T11:22:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:13.967+0000 7fa67402b700 1 -- 172.21.15.7:0/2047496770 wait complete. 2023-10-06T11:22:13.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 Processor -- start 2023-10-06T11:22:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 -- start start 2023-10-06T11:22:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa98c14fdf0 0x7fa98c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa98c0b07f0 0x7fa98c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c0ae860 con 0x7fa98c0cc190 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c157010 con 0x7fa98c14fdf0 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa990acb700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0aeb40 con 0x7fa98c0b07f0 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa98b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa98c14fdf0 0x7fa98c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa98b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa98c14fdf0 0x7fa98c1501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41456/0 (socket says 172.21.15.7:41456) 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3299059543 learned_addr learned my addr 172.21.15.7:0/3299059543 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.087+0000 7fa98bfff700 1 --2- 172.21.15.7:0/3299059543 >> v2:172.21.15.50:3300/0 conn(0x7fa98c0b07f0 0x7fa98c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3299059543 >> v2:172.21.15.50:3300/0 conn(0x7fa98c0b07f0 msgr2=0x7fa98c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98b7fe700 1 --2- 172.21.15.7:0/3299059543 >> v2:172.21.15.50:3300/0 conn(0x7fa98c0b07f0 0x7fa98c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3299059543 >> v1:172.21.15.50:6789/0 conn(0x7fa98c0cc190 legacy=0x7fa98c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3299059543 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0aee20 con 0x7fa98c14fdf0 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98b7fe700 1 --2- 172.21.15.7:0/3299059543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c14fdf0 0x7fa98c1501c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0cb8e0 tx=0x7fa97c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef7029679ba767a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98a7fc700 1 -- 172.21.15.7:0/3299059543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c009e70 con 0x7fa98c14fdf0 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98a7fc700 1 -- 172.21.15.7:0/3299059543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c00b860 con 0x7fa98c14fdf0 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa98a7fc700 1 -- 172.21.15.7:0/3299059543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00bc00 con 0x7fa98c14fdf0 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- 172.21.15.7:0/3299059543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c14fdf0 msgr2=0x7fa98c1501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 --2- 172.21.15.7:0/3299059543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c14fdf0 0x7fa98c1501c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0cb8e0 tx=0x7fa97c009250 comp rx=0 tx=0).stop 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- 172.21.15.7:0/3299059543 shutdown_connections 2023-10-06T11:22:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 --2- 172.21.15.7:0/3299059543 >> v2:172.21.15.50:3300/0 conn(0x7fa98c0b07f0 0x7fa98c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 --2- 172.21.15.7:0/3299059543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c14fdf0 0x7fa98c1501c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- 172.21.15.7:0/3299059543 >> 172.21.15.7:0/3299059543 conn(0x7fa98c0688a0 msgr2=0x7fa98c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- 172.21.15.7:0/3299059543 shutdown_connections 2023-10-06T11:22:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- 172.21.15.7:0/3299059543 wait complete. 2023-10-06T11:22:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 Processor -- start 2023-10-06T11:22:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.091+0000 7fa990acb700 1 -- start start 2023-10-06T11:22:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 0x7fa98c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa98c14fdf0 0x7fa98c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c5730 con 0x7fa98c0b07f0 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c7440 con 0x7fa98c14fdf0 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c14e310 con 0x7fa98c0cc190 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41466/0 (socket says 172.21.15.7:41466) 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3521608662 learned_addr learned my addr 172.21.15.7:0/3521608662 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98affd700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 0x7fa98c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98bfff700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa98c14fdf0 0x7fa98c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 msgr2=0x7fa98c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 0x7fa98c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa98c14fdf0 msgr2=0x7fa98c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa98c14fdf0 0x7fa98c0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 -- 172.21.15.7:0/3521608662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0094f0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98affd700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 0x7fa98c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa98b7fe700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af8a0 tx=0x7fa97c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00bbe0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c011420 con 0x7fa98c0b07f0 2023-10-06T11:22:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c0c6b50 con 0x7fa98c0b07f0 2023-10-06T11:22:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0117c0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.095+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c0c6df0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c06c710 con 0x7fa98c0b07f0 2023-10-06T11:22:14.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa97c011bf0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa9737fe700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 0x7fa978062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa97c07dfc0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa98affd700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 0x7fa978062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.099+0000 7fa98affd700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 0x7fa978062a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0ae080 tx=0x7fa980009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.111+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa97c0adbe0 con 0x7fa98c0b07f0 2023-10-06T11:22:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.275+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa98c0af100 con 0x7fa98c0b07f0 2023-10-06T11:22:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa9737fe700 1 -- 172.21.15.7:0/3521608662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7fa97c04e160 con 0x7fa98c0b07f0 2023-10-06T11:22:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:14.284 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 msgr2=0x7fa978062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 0x7fa978062a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0ae080 tx=0x7fa980009040 comp rx=0 tx=0).stop 2023-10-06T11:22:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 msgr2=0x7fa98c0c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af8a0 tx=0x7fa97c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 shutdown_connections 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa978060580 0x7fa978062a30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa98c14fdf0 0x7fa98c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa98c0cc190 0x7fa98c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 --2- 172.21.15.7:0/3521608662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98c0b07f0 0x7fa98c0c78d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.279+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 >> 172.21.15.7:0/3521608662 conn(0x7fa98c0688a0 msgr2=0x7fa98c0695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.283+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 shutdown_connections 2023-10-06T11:22:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.283+0000 7fa990acb700 1 -- 172.21.15.7:0/3521608662 wait complete. 2023-10-06T11:22:14.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:22:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 Processor -- start 2023-10-06T11:22:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 -- start start 2023-10-06T11:22:14.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5c780ccdc0 0x7f5c7806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7806e770 con 0x7f5c78159eb0 2023-10-06T11:22:14.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780ceb50 con 0x7f5c780ccdc0 2023-10-06T11:22:14.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7fb45700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780ce220 con 0x7f5c780cefe0 2023-10-06T11:22:14.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7d0e0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7e0e2700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5c780ccdc0 0x7f5c7806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7d0e0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43208/0 (socket says 172.21.15.7:43208) 2023-10-06T11:22:14.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7d8e1700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5c78159eb0 0x7f5c780cc6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:47306/0 (socket says 172.21.15.7:47306) 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.403+0000 7f5c7d0e0700 1 -- 172.21.15.7:0/3223837443 learned_addr learned my addr 172.21.15.7:0/3223837443 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7d0e0700 1 -- 172.21.15.7:0/3223837443 >> v2:172.21.15.50:3300/0 conn(0x7f5c780ccdc0 msgr2=0x7f5c7806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7d0e0700 1 --2- 172.21.15.7:0/3223837443 >> v2:172.21.15.50:3300/0 conn(0x7f5c780ccdc0 0x7f5c7806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7d0e0700 1 -- 172.21.15.7:0/3223837443 >> v1:172.21.15.50:6789/0 conn(0x7f5c78159eb0 legacy=0x7f5c780cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7d0e0700 1 -- 172.21.15.7:0/3223837443 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780ce500 con 0x7f5c780cefe0 2023-10-06T11:22:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7d0e0700 1 --2- 172.21.15.7:0/3223837443 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c680099b0 tx=0x7f5c68009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a010bddcd61cd90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7c8df700 1 -- 172.21.15.7:0/3223837443 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68009ea0 con 0x7f5c780cefe0 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7c8df700 1 -- 172.21.15.7:0/3223837443 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800b860 con 0x7f5c780cefe0 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7c8df700 1 -- 172.21.15.7:0/3223837443 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800bc00 con 0x7f5c780cefe0 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 -- 172.21.15.7:0/3223837443 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 msgr2=0x7f5c7806d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/3223837443 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c680099b0 tx=0x7f5c68009250 comp rx=0 tx=0).stop 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 -- 172.21.15.7:0/3223837443 shutdown_connections 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/3223837443 >> v2:172.21.15.50:3300/0 conn(0x7f5c780ccdc0 0x7f5c7806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/3223837443 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c780cefe0 0x7f5c7806d9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 -- 172.21.15.7:0/3223837443 >> 172.21.15.7:0/3223837443 conn(0x7f5c78067e30 msgr2=0x7f5c78068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 -- 172.21.15.7:0/3223837443 shutdown_connections 2023-10-06T11:22:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.407+0000 7f5c7fb45700 1 -- 172.21.15.7:0/3223837443 wait complete. 2023-10-06T11:22:14.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 Processor -- start 2023-10-06T11:22:14.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 -- start start 2023-10-06T11:22:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c780cefe0 0x7f5c780c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 0x7f5c7815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c4f20 con 0x7f5c780ccdc0 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781578a0 con 0x7f5c780cefe0 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7fb45700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c4310 con 0x7f5c78159eb0 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7d8e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7d8e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41480/0 (socket says 172.21.15.7:41480) 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7d8e1700 1 -- 172.21.15.7:0/1145016406 learned_addr learned my addr 172.21.15.7:0/1145016406 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 0x7f5c7815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7d0e0700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c780cefe0 0x7f5c780c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 -- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c780cefe0 msgr2=0x7f5c780c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c780cefe0 0x7f5c780c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 -- 172.21.15.7:0/1145016406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 msgr2=0x7f5c78157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 -- 172.21.15.7:0/1145016406 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680094f0 con 0x7f5c78159eb0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7d8e1700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c7e0e2700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 0x7f5c7815c7d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e900 tx=0x7f5c700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70013680 con 0x7f5c78159eb0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.411+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70013820 con 0x7f5c78159eb0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780c45f0 con 0x7f5c78159eb0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70013be0 con 0x7f5c78159eb0 2023-10-06T11:22:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780c4bb0 con 0x7f5c78159eb0 2023-10-06T11:22:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c780c5090 con 0x7f5c78159eb0 2023-10-06T11:22:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c700123d0 con 0x7f5c78159eb0 2023-10-06T11:22:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c6dffb700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 0x7f5c5c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5c7007ca10 con 0x7f5c78159eb0 2023-10-06T11:22:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.415+0000 7f5c7d8e1700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 0x7f5c5c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.419+0000 7f5c7d8e1700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 0x7f5c5c062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c74002230 tx=0x7f5c74001b90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.431+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c7004cb10 con 0x7f5c78159eb0 2023-10-06T11:22:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:14.631+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5c7804be00 con 0x7f5c78159eb0 2023-10-06T11:22:15.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.231+0000 7f5c6dffb700 1 -- 172.21.15.7:0/1145016406 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5c7007ae60 con 0x7f5c78159eb0 2023-10-06T11:22:15.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 msgr2=0x7f5c5c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 0x7f5c5c062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c74002230 tx=0x7f5c74001b90 comp rx=0 tx=0).stop 2023-10-06T11:22:15.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 msgr2=0x7f5c7815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 0x7f5c7815c7d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e900 tx=0x7f5c700091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 shutdown_connections 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5c5c060550 0x7f5c5c062a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5c78159eb0 0x7f5c7815c7d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c780cefe0 0x7f5c780c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 --2- 172.21.15.7:0/1145016406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780ccdc0 0x7f5c78157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 >> 172.21.15.7:0/1145016406 conn(0x7f5c78067e30 msgr2=0x7f5c7806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 shutdown_connections 2023-10-06T11:22:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.235+0000 7f5c7fb45700 1 -- 172.21.15.7:0/1145016406 wait complete. 2023-10-06T11:22:15.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:22:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.367+0000 7fa41358e700 1 Processor -- start 2023-10-06T11:22:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.367+0000 7fa41358e700 1 -- start start 2023-10-06T11:22:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41358e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41358e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c0c8030 con 0x7fa40c069de0 2023-10-06T11:22:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41358e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c0cab80 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41358e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0c79e0 con 0x7fa40c155220 2023-10-06T11:22:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa410b29700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa40c069de0 0x7fa40c0c9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54740/0 (socket says 172.21.15.7:54740) 2023-10-06T11:22:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa410b29700 1 -- 172.21.15.7:0/3151698314 learned_addr learned my addr 172.21.15.7:0/3151698314 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41132a700 1 --2- 172.21.15.7:0/3151698314 >> v2:172.21.15.50:3300/0 conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218988113 0 0) 0x7fa40c0c8030 con 0x7fa40c069de0 2023-10-06T11:22:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f8003570 con 0x7fa40c069de0 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41132a700 1 -- 172.21.15.7:0/3151698314 >> v1:172.21.15.50:6789/0 conn(0x7fa40c0ca5d0 legacy=0x7fa40c0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389297286 0 0) 0x7fa40c0cab80 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41132a700 1 -- 172.21.15.7:0/3151698314 >> v1:172.21.15.7:6789/0 conn(0x7fa40c069de0 legacy=0x7fa40c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41132a700 1 -- 172.21.15.7:0/3151698314 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0c7cc0 con 0x7fa40c155220 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa41132a700 1 --2- 172.21.15.7:0/3151698314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4080098e0 tx=0x7fa4080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d459807a2ea13d38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800b640 con 0x7fa40c155220 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800b7a0 con 0x7fa40c155220 2023-10-06T11:22:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.371+0000 7fa403fff700 1 -- 172.21.15.7:0/3151698314 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800bb40 con 0x7fa40c155220 2023-10-06T11:22:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- 172.21.15.7:0/3151698314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 msgr2=0x7fa40c0c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 --2- 172.21.15.7:0/3151698314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4080098e0 tx=0x7fa4080091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- 172.21.15.7:0/3151698314 shutdown_connections 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 --2- 172.21.15.7:0/3151698314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- 172.21.15.7:0/3151698314 >> 172.21.15.7:0/3151698314 conn(0x7fa40c0cc2f0 msgr2=0x7fa40c0cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- 172.21.15.7:0/3151698314 shutdown_connections 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- 172.21.15.7:0/3151698314 wait complete. 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 Processor -- start 2023-10-06T11:22:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- start start 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa40c069de0 0x7fa40c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c158340 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c151730 con 0x7fa40c155220 2023-10-06T11:22:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41358e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c16d190 con 0x7fa40c069de0 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa410b29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa410b29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41486/0 (socket says 172.21.15.7:41486) 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa410b29700 1 -- 172.21.15.7:0/1551075367 learned_addr learned my addr 172.21.15.7:0/1551075367 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa41132a700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa40c069de0 0x7fa40c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.375+0000 7fa411b2b700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 -- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa40c069de0 msgr2=0x7fa40c157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa40c069de0 0x7fa40c157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 -- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 msgr2=0x7fa40c0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 -- 172.21.15.7:0/1551075367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009420 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa411b2b700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa410b29700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0011e0 tx=0x7fa3fc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc005b30 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc005c90 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c156780 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc007320 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c16cbd0 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.379+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c06eb10 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.383+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa3fc007690 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.383+0000 7fa4017fa700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 0x7fa3f0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.383+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa3fc07c100 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.383+0000 7fa41132a700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 0x7fa3f0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.383+0000 7fa41132a700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 0x7fa3f0062a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4080093f0 tx=0x7fa408017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.391+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa40c06eb10 con 0x7fa40c0ca5d0 2023-10-06T11:22:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:15.587+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa40c06eb10 con 0x7fa40c0ca5d0 2023-10-06T11:22:16.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.247+0000 7fa4017fa700 1 -- 172.21.15.7:0/1551075367 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa40c06eb10 con 0x7fa40c0ca5d0 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 msgr2=0x7fa3f0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 0x7fa3f0062a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4080093f0 tx=0x7fa408017000 comp rx=0 tx=0).stop 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 msgr2=0x7fa40c16c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0011e0 tx=0x7fa3fc007df0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 shutdown_connections 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa3f0060580 0x7fa3f0062a30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa40c155220 0x7fa40c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa40c0ca5d0 0x7fa40c16c250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 --2- 172.21.15.7:0/1551075367 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa40c069de0 0x7fa40c157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 >> 172.21.15.7:0/1551075367 conn(0x7fa40c0cc2f0 msgr2=0x7fa40c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:16.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 shutdown_connections 2023-10-06T11:22:16.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.255+0000 7fa41358e700 1 -- 172.21.15.7:0/1551075367 wait complete. 2023-10-06T11:22:16.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 Processor -- start 2023-10-06T11:22:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 -- start start 2023-10-06T11:22:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91e00afb00 0x7f91e00afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00ca0a0 con 0x7f91e00ade50 2023-10-06T11:22:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e0157230 con 0x7f91e00ca470 2023-10-06T11:22:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.383+0000 7f91e4884700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00b0410 con 0x7f91e00afb00 2023-10-06T11:22:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91e00afb00 0x7f91e00afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91e00afb00 0x7f91e00afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41490/0 (socket says 172.21.15.7:41490) 2023-10-06T11:22:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 -- 172.21.15.7:0/2059992573 learned_addr learned my addr 172.21.15.7:0/2059992573 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ddd9b700 1 -- 172.21.15.7:0/2059992573 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421197529 0 0) 0x7f91e0157230 con 0x7f91e00ca470 2023-10-06T11:22:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 -- 172.21.15.7:0/2059992573 >> v1:172.21.15.50:6789/0 conn(0x7f91e00ca470 legacy=0x7f91e00ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 -- 172.21.15.7:0/2059992573 >> v1:172.21.15.7:6789/0 conn(0x7f91e00ade50 legacy=0x7f91e0160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 -- 172.21.15.7:0/2059992573 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e0157510 con 0x7f91e00afb00 2023-10-06T11:22:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ded9d700 1 --2- 172.21.15.7:0/2059992573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00afb00 0x7f91e00afed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009f20 tx=0x7f91c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02a5a67188398aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ddd9b700 1 -- 172.21.15.7:0/2059992573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800e070 con 0x7f91e00afb00 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ddd9b700 1 -- 172.21.15.7:0/2059992573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800ba80 con 0x7f91e00afb00 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91ddd9b700 1 -- 172.21.15.7:0/2059992573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80092e0 con 0x7f91e00afb00 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- 172.21.15.7:0/2059992573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00afb00 msgr2=0x7f91e00afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 --2- 172.21.15.7:0/2059992573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00afb00 0x7f91e00afed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009f20 tx=0x7f91c8009840 comp rx=0 tx=0).stop 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- 172.21.15.7:0/2059992573 shutdown_connections 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 --2- 172.21.15.7:0/2059992573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00afb00 0x7f91e00afed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- 172.21.15.7:0/2059992573 >> 172.21.15.7:0/2059992573 conn(0x7f91e006bd70 msgr2=0x7f91e006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- 172.21.15.7:0/2059992573 shutdown_connections 2023-10-06T11:22:16.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- 172.21.15.7:0/2059992573 wait complete. 2023-10-06T11:22:16.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 Processor -- start 2023-10-06T11:22:16.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.387+0000 7f91e4884700 1 -- start start 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 0x7f91e00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91e00afb00 0x7f91e00c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cea80 con 0x7f91e00ca470 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c6da0 con 0x7f91e00ade50 2023-10-06T11:22:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01558d0 con 0x7f91e00afb00 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41506/0 (socket says 172.21.15.7:41506) 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 -- 172.21.15.7:0/2855787896 learned_addr learned my addr 172.21.15.7:0/2855787896 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91ded9d700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 0x7f91e00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91de59c700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91e00afb00 0x7f91e00c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 -- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91e00afb00 msgr2=0x7f91e00c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91e00afb00 0x7f91e00c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 -- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 msgr2=0x7f91e00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 0x7f91e00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 -- 172.21.15.7:0/2855787896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009ae0 con 0x7f91e00ca470 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91ded9d700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 0x7f91e00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91df59e700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c4f0 tx=0x7f91d8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8011590 con 0x7f91e00ca470 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8011710 con 0x7f91e00ca470 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e00c5ad0 con 0x7f91e00ca470 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8011ad0 con 0x7f91e00ca470 2023-10-06T11:22:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e00c5d80 con 0x7f91e00ca470 2023-10-06T11:22:16.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.391+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e006ec60 con 0x7f91e00ca470 2023-10-06T11:22:16.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.395+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f91d801b3b0 con 0x7f91e00ca470 2023-10-06T11:22:16.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.395+0000 7f91d6ffd700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 0x7f91c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.395+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f91d807c0f0 con 0x7f91e00ca470 2023-10-06T11:22:16.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.395+0000 7f91ded9d700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 0x7f91c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.395+0000 7f91ded9d700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 0x7f91c4062a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ab0 tx=0x7f91c8018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.399+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f91d804c1f0 con 0x7f91e00ca470 2023-10-06T11:22:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.595+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91e004b620 con 0x7f91e00ca470 2023-10-06T11:22:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.595+0000 7f91d6ffd700 1 -- 172.21.15.7:0/2855787896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f91d804f810 con 0x7f91e00ca470 2023-10-06T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 msgr2=0x7f91c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 0x7f91c4062a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ab0 tx=0x7f91c8018000 comp rx=0 tx=0).stop 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 msgr2=0x7f91e0168e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c4f0 tx=0x7f91d8007f50 comp rx=0 tx=0).stop 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 shutdown_connections 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f91c4060550 0x7f91c4062a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91e00ca470 0x7f91e0168e50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91e00afb00 0x7f91e00c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 --2- 172.21.15.7:0/2855787896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e00ade50 0x7f91e00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 >> 172.21.15.7:0/2855787896 conn(0x7f91e006bd70 msgr2=0x7f91e0159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 shutdown_connections 2023-10-06T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.599+0000 7f91e4884700 1 -- 172.21.15.7:0/2855787896 wait complete. 2023-10-06T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:16.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:16.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 Processor -- start 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 -- start start 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99415b5a0 con 0x7fc9940cbe20 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99415a990 con 0x7fc9940c07c0 2023-10-06T11:22:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9992a9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99415ac70 con 0x7fc9941567e0 2023-10-06T11:22:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9927fc700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43236/0 (socket says 172.21.15.7:43236) 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9927fc700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57628/0 (socket says 172.21.15.7:57628) 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 -- 172.21.15.7:0/2780161385 learned_addr learned my addr 172.21.15.7:0/2780161385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 -- 172.21.15.7:0/2780161385 >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 msgr2=0x7fc9940cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 --2- 172.21.15.7:0/2780161385 >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 -- 172.21.15.7:0/2780161385 >> v1:172.21.15.50:6789/0 conn(0x7fc9940cbe20 legacy=0x7fc9940ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 -- 172.21.15.7:0/2780161385 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99415af50 con 0x7fc9941567e0 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9927fc700 1 --2- 172.21.15.7:0/2780161385 >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.715+0000 7fc9937fe700 1 --2- 172.21.15.7:0/2780161385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0099b0 tx=0x7fc98c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7143cd42efa45acb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc991ffb700 1 -- 172.21.15.7:0/2780161385 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c009ea0 con 0x7fc9941567e0 2023-10-06T11:22:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc991ffb700 1 -- 172.21.15.7:0/2780161385 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00b860 con 0x7fc9941567e0 2023-10-06T11:22:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc991ffb700 1 -- 172.21.15.7:0/2780161385 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00bc00 con 0x7fc9941567e0 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- 172.21.15.7:0/2780161385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 msgr2=0x7fc994156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2780161385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0099b0 tx=0x7fc98c009250 comp rx=0 tx=0).stop 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- 172.21.15.7:0/2780161385 shutdown_connections 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2780161385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9941567e0 0x7fc994156c60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2780161385 >> v2:172.21.15.50:3300/0 conn(0x7fc9940c07c0 0x7fc9940cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- 172.21.15.7:0/2780161385 >> 172.21.15.7:0/2780161385 conn(0x7fc994066840 msgr2=0x7fc994066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- 172.21.15.7:0/2780161385 shutdown_connections 2023-10-06T11:22:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- 172.21.15.7:0/2780161385 wait complete. 2023-10-06T11:22:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 Processor -- start 2023-10-06T11:22:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 -- start start 2023-10-06T11:22:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.719+0000 7fc9992a9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9940cbe20 0x7fc99416a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9941567e0 0x7fc99416aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940c8210 con 0x7fc9940c07c0 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940aec60 con 0x7fc9941567e0 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940c7c50 con 0x7fc9940cbe20 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41512/0 (socket says 172.21.15.7:41512) 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9937fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9941567e0 0x7fc99416aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9927fc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9940cbe20 0x7fc99416a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 -- 172.21.15.7:0/2099014230 learned_addr learned my addr 172.21.15.7:0/2099014230 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 -- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9940cbe20 msgr2=0x7fc99416a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9940cbe20 0x7fc99416a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 -- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9941567e0 msgr2=0x7fc99416aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9941567e0 0x7fc99416aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 -- 172.21.15.7:0/2099014230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988009a20 con 0x7fc9940c07c0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc992ffd700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c7c0 tx=0x7fc984007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011590 con 0x7fc9940c07c0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984011710 con 0x7fc9940c07c0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c0094f0 con 0x7fc9940c07c0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011ad0 con 0x7fc9940c07c0 2023-10-06T11:22:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9940ae8f0 con 0x7fc9940c07c0 2023-10-06T11:22:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.723+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99404be00 con 0x7fc9940c07c0 2023-10-06T11:22:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.727+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc98401b3b0 con 0x7fc9940c07c0 2023-10-06T11:22:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.727+0000 7fc97affd700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 0x7fc980062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.727+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc98407bfe0 con 0x7fc9940c07c0 2023-10-06T11:22:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.727+0000 7fc9927fc700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 0x7fc980062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.727+0000 7fc9927fc700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 0x7fc980062a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fd0 tx=0x7fc98800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.735+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc98404c120 con 0x7fc9940c07c0 2023-10-06T11:22:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.899+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc99415b230 con 0x7fc9940c07c0 2023-10-06T11:22:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.899+0000 7fc97affd700 1 -- 172.21.15.7:0/2099014230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7fc98404f740 con 0x7fc9940c07c0 2023-10-06T11:22:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:16.905 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 msgr2=0x7fc980062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 0x7fc980062a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fd0 tx=0x7fc98800b040 comp rx=0 tx=0).stop 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 msgr2=0x7fc9940c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c7c0 tx=0x7fc984007f40 comp rx=0 tx=0).stop 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 shutdown_connections 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc980060550 0x7fc980062a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9941567e0 0x7fc99416aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc9940cbe20 0x7fc99416a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 --2- 172.21.15.7:0/2099014230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9940c07c0 0x7fc9940c7300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 >> 172.21.15.7:0/2099014230 conn(0x7fc994066840 msgr2=0x7fc994158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 shutdown_connections 2023-10-06T11:22:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:16.903+0000 7fc9992a9700 1 -- 172.21.15.7:0/2099014230 wait complete. 2023-10-06T11:22:16.919 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:22:16.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:22:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.019+0000 7fd0910a2700 1 Processor -- start 2023-10-06T11:22:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.019+0000 7fd0910a2700 1 -- start start 2023-10-06T11:22:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd08c158b40 0x7fd08c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c06e310 con 0x7fd08c0cb1e0 2023-10-06T11:22:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c06e7a0 con 0x7fd08c0ccc60 2023-10-06T11:22:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c06d720 con 0x7fd08c158b40 2023-10-06T11:22:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd08c0cb1e0 0x7fd08c0cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54762/0 (socket says 172.21.15.7:54762) 2023-10-06T11:22:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08b7fe700 1 -- 172.21.15.7:0/1214681408 learned_addr learned my addr 172.21.15.7:0/1214681408 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08bfff700 1 --2- 172.21.15.7:0/1214681408 >> v2:172.21.15.50:3300/0 conn(0x7fd08c158b40 0x7fd08c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719449091 0 0) 0x7fd08c06e310 con 0x7fd08c0cb1e0 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd078003570 con 0x7fd08c0cb1e0 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548756708 0 0) 0x7fd08c06e7a0 con 0x7fd08c0ccc60 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08c06e310 con 0x7fd08c0ccc60 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08bfff700 1 -- 172.21.15.7:0/1214681408 >> v1:172.21.15.50:6789/0 conn(0x7fd08c0ccc60 legacy=0x7fd08c069720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08bfff700 1 -- 172.21.15.7:0/1214681408 >> v1:172.21.15.7:6789/0 conn(0x7fd08c0cb1e0 legacy=0x7fd08c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08bfff700 1 -- 172.21.15.7:0/1214681408 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c06d9e0 con 0x7fd08c158b40 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08bfff700 1 --2- 172.21.15.7:0/1214681408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c158b40 0x7fd08c0caca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0098e0 tx=0x7fd07c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c58cc050212dd5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011630 con 0x7fd08c158b40 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c011790 con 0x7fd08c158b40 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd08affd700 1 -- 172.21.15.7:0/1214681408 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011b30 con 0x7fd08c158b40 2023-10-06T11:22:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- 172.21.15.7:0/1214681408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c158b40 msgr2=0x7fd08c0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1214681408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c158b40 0x7fd08c0caca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0098e0 tx=0x7fd07c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- 172.21.15.7:0/1214681408 shutdown_connections 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1214681408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c158b40 0x7fd08c0caca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- 172.21.15.7:0/1214681408 >> 172.21.15.7:0/1214681408 conn(0x7fd08c067e30 msgr2=0x7fd08c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- 172.21.15.7:0/1214681408 shutdown_connections 2023-10-06T11:22:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.023+0000 7fd0910a2700 1 -- 172.21.15.7:0/1214681408 wait complete. 2023-10-06T11:22:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 Processor -- start 2023-10-06T11:22:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 -- start start 2023-10-06T11:22:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd08c0cb1e0 0x7fd08c1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c0ccc60 0x7fd08c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b4200 con 0x7fd08c158b40 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c140930 con 0x7fd08c0ccc60 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0910a2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b35f0 con 0x7fd08c0cb1e0 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41522/0 (socket says 172.21.15.7:41522) 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 -- 172.21.15.7:0/1888670128 learned_addr learned my addr 172.21.15.7:0/1888670128 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd08bfff700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd08c0cb1e0 0x7fd08c1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd08b7fe700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c0ccc60 0x7fd08c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 -- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd08c0cb1e0 msgr2=0x7fd08c1403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd08c0cb1e0 0x7fd08c1403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 -- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c0ccc60 msgr2=0x7fd08c157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c0ccc60 0x7fd08c157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 -- 172.21.15.7:0/1888670128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c009420 con 0x7fd08c158b40 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.027+0000 7fd0908a1700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c940 tx=0x7fd0840027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084012630 con 0x7fd08c158b40 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0840127b0 con 0x7fd08c158b40 2023-10-06T11:22:17.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c0b38d0 con 0x7fd08c158b40 2023-10-06T11:22:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084012b70 con 0x7fd08c158b40 2023-10-06T11:22:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c0b3e90 con 0x7fd08c158b40 2023-10-06T11:22:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd08401b400 con 0x7fd08c158b40 2023-10-06T11:22:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c06ca60 con 0x7fd08c158b40 2023-10-06T11:22:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 0x7fd06c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd08407c260 con 0x7fd08c158b40 2023-10-06T11:22:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.031+0000 7fd08bfff700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 0x7fd06c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.035+0000 7fd08bfff700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 0x7fd06c062a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0093f0 tx=0x7fd07c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.039+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd084081060 con 0x7fd08c158b40 2023-10-06T11:22:17.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.199+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd08c06dcc0 con 0x7fd06c060550 2023-10-06T11:22:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.219+0000 7fd083fff700 1 -- 172.21.15.7:0/1888670128 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367453 (secure 0 0 0) 0x7fd08c06dcc0 con 0x7fd06c060550 2023-10-06T11:22:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 msgr2=0x7fd06c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 0x7fd06c062a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0093f0 tx=0x7fd07c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 msgr2=0x7fd08c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c940 tx=0x7fd0840027a0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 shutdown_connections 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd06c060550 0x7fd06c062a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c158b40 0x7fd08c0cde00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd08c0ccc60 0x7fd08c157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 --2- 172.21.15.7:0/1888670128 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd08c0cb1e0 0x7fd08c1403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 >> 172.21.15.7:0/1888670128 conn(0x7fd08c067e30 msgr2=0x7fd08c15ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 shutdown_connections 2023-10-06T11:22:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.227+0000 7fd0910a2700 1 -- 172.21.15.7:0/1888670128 wait complete. 2023-10-06T11:22:17.234 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:22:17.247 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-06T11:22:15.816921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414376,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752560344,"statfs":{"total":773094113280,"available":770621792256,"internally_reserved":0,"allocated":3383296,"data_stored":2065368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005022"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642766+0000","last_change":"2023-10-06T11:22:01.634221+0000","last_active":"2023-10-06T11:22:01.642766+0000","last_peered":"2023-10-06T11:22:01.642766+0000","last_clean":"2023-10-06T11:22:01.642766+0000","last_became_active":"2023-10-06T11:22:01.633727+0000","last_became_peered":"2023-10-06T11:22:01.633727+0000","last_unstale":"2023-10-06T11:22:01.642766+0000","last_undegraded":"2023-10-06T11:22:01.642766+0000","last_fullsized":"2023-10-06T11:22:01.642766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627705+0000","last_change":"2023-10-06T11:21:56.568541+0000","last_active":"2023-10-06T11:22:01.627705+0000","last_peered":"2023-10-06T11:22:01.627705+0000","last_clean":"2023-10-06T11:22:01.627705+0000","last_became_active":"2023-10-06T11:21:56.567104+0000","last_became_peered":"2023-10-06T11:21:56.567104+0000","last_unstale":"2023-10-06T11:22:01.627705+0000","last_undegraded":"2023-10-06T11:22:01.627705+0000","last_fullsized":"2023-10-06T11:22:01.627705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612705+0000","last_change":"2023-10-06T11:21:57.566064+0000","last_active":"2023-10-06T11:22:01.612705+0000","last_peered":"2023-10-06T11:22:01.612705+0000","last_clean":"2023-10-06T11:22:01.612705+0000","last_became_active":"2023-10-06T11:21:57.565589+0000","last_became_peered":"2023-10-06T11:21:57.565589+0000","last_unstale":"2023-10-06T11:22:01.612705+0000","last_undegraded":"2023-10-06T11:22:01.612705+0000","last_fullsized":"2023-10-06T11:22:01.612705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645194+0000","last_change":"2023-10-06T11:22:01.637184+0000","last_active":"2023-10-06T11:22:01.645194+0000","last_peered":"2023-10-06T11:22:01.645194+0000","last_clean":"2023-10-06T11:22:01.645194+0000","last_became_active":"2023-10-06T11:22:01.636773+0000","last_became_peered":"2023-10-06T11:22:01.636773+0000","last_unstale":"2023-10-06T11:22:01.645194+0000","last_undegraded":"2023-10-06T11:22:01.645194+0000","last_fullsized":"2023-10-06T11:22:01.645194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:41.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617318+0000","last_change":"2023-10-06T11:21:56.565259+0000","last_active":"2023-10-06T11:22:01.617318+0000","last_peered":"2023-10-06T11:22:01.617318+0000","last_clean":"2023-10-06T11:22:01.617318+0000","last_became_active":"2023-10-06T11:21:56.564927+0000","last_became_peered":"2023-10-06T11:21:56.564927+0000","last_unstale":"2023-10-06T11:22:01.617318+0000","last_undegraded":"2023-10-06T11:22:01.617318+0000","last_fullsized":"2023-10-06T11:22:01.617318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617577+0000","last_change":"2023-10-06T11:21:57.562278+0000","last_active":"2023-10-06T11:22:01.617577+0000","last_peered":"2023-10-06T11:22:01.617577+0000","last_clean":"2023-10-06T11:22:01.617577+0000","last_became_active":"2023-10-06T11:21:57.561972+0000","last_became_peered":"2023-10-06T11:21:57.561972+0000","last_unstale":"2023-10-06T11:22:01.617577+0000","last_undegraded":"2023-10-06T11:22:01.617577+0000","last_fullsized":"2023-10-06T11:22:01.617577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641228+0000","last_change":"2023-10-06T11:22:01.637805+0000","last_active":"2023-10-06T11:22:01.641228+0000","last_peered":"2023-10-06T11:22:01.641228+0000","last_clean":"2023-10-06T11:22:01.641228+0000","last_became_active":"2023-10-06T11:22:01.637404+0000","last_became_peered":"2023-10-06T11:22:01.637404+0000","last_unstale":"2023-10-06T11:22:01.641228+0000","last_undegraded":"2023-10-06T11:22:01.641228+0000","last_fullsized":"2023-10-06T11:22:01.641228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621633+0000","last_change":"2023-10-06T11:21:56.578707+0000","last_active":"2023-10-06T11:22:01.621633+0000","last_peered":"2023-10-06T11:22:01.621633+0000","last_clean":"2023-10-06T11:22:01.621633+0000","last_became_active":"2023-10-06T11:21:56.578411+0000","last_became_peered":"2023-10-06T11:21:56.578411+0000","last_unstale":"2023-10-06T11:22:01.621633+0000","last_undegraded":"2023-10-06T11:22:01.621633+0000","last_fullsized":"2023-10-06T11:22:01.621633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610166+0000","last_change":"2023-10-06T11:21:57.570591+0000","last_active":"2023-10-06T11:22:01.610166+0000","last_peered":"2023-10-06T11:22:01.610166+0000","last_clean":"2023-10-06T11:22:01.610166+0000","last_became_active":"2023-10-06T11:21:57.570166+0000","last_became_peered":"2023-10-06T11:21:57.570166+0000","last_unstale":"2023-10-06T11:22:01.610166+0000","last_undegraded":"2023-10-06T11:22:01.610166+0000","last_fullsized":"2023-10-06T11:22:01.610166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644696+0000","last_change":"2023-10-06T11:22:01.638015+0000","last_active":"2023-10-06T11:22:01.644696+0000","last_peered":"2023-10-06T11:22:01.644696+0000","last_clean":"2023-10-06T11:22:01.644696+0000","last_became_active":"2023-10-06T11:22:01.637477+0000","last_became_peered":"2023-10-06T11:22:01.637477+0000","last_unstale":"2023-10-06T11:22:01.644696+0000","last_undegraded":"2023-10-06T11:22:01.644696+0000","last_fullsized":"2023-10-06T11:22:01.644696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:56.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621240+0000","last_change":"2023-10-06T11:21:56.571616+0000","last_active":"2023-10-06T11:22:01.621240+0000","last_peered":"2023-10-06T11:22:01.621240+0000","last_clean":"2023-10-06T11:22:01.621240+0000","last_became_active":"2023-10-06T11:21:56.571289+0000","last_became_peered":"2023-10-06T11:21:56.571289+0000","last_unstale":"2023-10-06T11:22:01.621240+0000","last_undegraded":"2023-10-06T11:22:01.621240+0000","last_fullsized":"2023-10-06T11:22:01.621240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611658+0000","last_change":"2023-10-06T11:21:57.575085+0000","last_active":"2023-10-06T11:22:01.611658+0000","last_peered":"2023-10-06T11:22:01.611658+0000","last_clean":"2023-10-06T11:22:01.611658+0000","last_became_active":"2023-10-06T11:21:57.574716+0000","last_became_peered":"2023-10-06T11:21:57.574716+0000","last_unstale":"2023-10-06T11:22:01.611658+0000","last_undegraded":"2023-10-06T11:22:01.611658+0000","last_fullsized":"2023-10-06T11:22:01.611658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643969+0000","last_change":"2023-10-06T11:22:01.638673+0000","last_active":"2023-10-06T11:22:01.643969+0000","last_peered":"2023-10-06T11:22:01.643969+0000","last_clean":"2023-10-06T11:22:01.643969+0000","last_became_active":"2023-10-06T11:22:01.635630+0000","last_became_peered":"2023-10-06T11:22:01.635630+0000","last_unstale":"2023-10-06T11:22:01.643969+0000","last_undegraded":"2023-10-06T11:22:01.643969+0000","last_fullsized":"2023-10-06T11:22:01.643969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626800+0000","last_change":"2023-10-06T11:21:56.572349+0000","last_active":"2023-10-06T11:22:01.626800+0000","last_peered":"2023-10-06T11:22:01.626800+0000","last_clean":"2023-10-06T11:22:01.626800+0000","last_became_active":"2023-10-06T11:21:56.571848+0000","last_became_peered":"2023-10-06T11:21:56.571848+0000","last_unstale":"2023-10-06T11:22:01.626800+0000","last_undegraded":"2023-10-06T11:22:01.626800+0000","last_fullsized":"2023-10-06T11:22:01.626800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607694+0000","last_change":"2023-10-06T11:21:57.580595+0000","last_active":"2023-10-06T11:22:01.607694+0000","last_peered":"2023-10-06T11:22:01.607694+0000","last_clean":"2023-10-06T11:22:01.607694+0000","last_became_active":"2023-10-06T11:21:57.580286+0000","last_became_peered":"2023-10-06T11:21:57.580286+0000","last_unstale":"2023-10-06T11:22:01.607694+0000","last_undegraded":"2023-10-06T11:22:01.607694+0000","last_fullsized":"2023-10-06T11:22:01.607694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:04.725222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642661+0000","last_change":"2023-10-06T11:22:01.632283+0000","last_active":"2023-10-06T11:22:01.642661+0000","last_peered":"2023-10-06T11:22:01.642661+0000","last_clean":"2023-10-06T11:22:01.642661+0000","last_became_active":"2023-10-06T11:22:01.628222+0000","last_became_peered":"2023-10-06T11:22:01.628222+0000","last_unstale":"2023-10-06T11:22:01.642661+0000","last_undegraded":"2023-10-06T11:22:01.642661+0000","last_fullsized":"2023-10-06T11:22:01.642661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:32.168166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610453+0000","last_change":"2023-10-06T11:21:56.571031+0000","last_active":"2023-10-06T11:22:01.610453+0000","last_peered":"2023-10-06T11:22:01.610453+0000","last_clean":"2023-10-06T11:22:01.610453+0000","last_became_active":"2023-10-06T11:21:56.570724+0000","last_became_peered":"2023-10-06T11:21:56.570724+0000","last_unstale":"2023-10-06T11:22:01.610453+0000","last_undegraded":"2023-10-06T11:22:01.610453+0000","last_fullsized":"2023-10-06T11:22:01.610453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616088+0000","last_change":"2023-10-06T11:21:57.562757+0000","last_active":"2023-10-06T11:22:01.616088+0000","last_peered":"2023-10-06T11:22:01.616088+0000","last_clean":"2023-10-06T11:22:01.616088+0000","last_became_active":"2023-10-06T11:21:57.562240+0000","last_became_peered":"2023-10-06T11:21:57.562240+0000","last_unstale":"2023-10-06T11:22:01.616088+0000","last_undegraded":"2023-10-06T11:22:01.616088+0000","last_fullsized":"2023-10-06T11:22:01.616088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.636503+0000","last_change":"2023-10-06T11:22:01.635655+0000","last_active":"2023-10-06T11:22:01.636503+0000","last_peered":"2023-10-06T11:22:01.636503+0000","last_clean":"2023-10-06T11:22:01.636503+0000","last_became_active":"2023-10-06T11:22:01.635167+0000","last_became_peered":"2023-10-06T11:22:01.635167+0000","last_unstale":"2023-10-06T11:22:01.636503+0000","last_undegraded":"2023-10-06T11:22:01.636503+0000","last_fullsized":"2023-10-06T11:22:01.636503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:16.080135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617232+0000","last_change":"2023-10-06T11:21:56.569415+0000","last_active":"2023-10-06T11:22:01.617232+0000","last_peered":"2023-10-06T11:22:01.617232+0000","last_clean":"2023-10-06T11:22:01.617232+0000","last_became_active":"2023-10-06T11:21:56.569132+0000","last_became_peered":"2023-10-06T11:21:56.569132+0000","last_unstale":"2023-10-06T11:22:01.617232+0000","last_undegraded":"2023-10-06T11:22:01.617232+0000","last_fullsized":"2023-10-06T11:22:01.617232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627493+0000","last_change":"2023-10-06T11:21:57.566844+0000","last_active":"2023-10-06T11:22:01.627493+0000","last_peered":"2023-10-06T11:22:01.627493+0000","last_clean":"2023-10-06T11:22:01.627493+0000","last_became_active":"2023-10-06T11:21:57.566564+0000","last_became_peered":"2023-10-06T11:21:57.566564+0000","last_unstale":"2023-10-06T11:22:01.627493+0000","last_undegraded":"2023-10-06T11:22:01.627493+0000","last_fullsized":"2023-10-06T11:22:01.627493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640584+0000","last_change":"2023-10-06T11:22:01.639744+0000","last_active":"2023-10-06T11:22:01.640584+0000","last_peered":"2023-10-06T11:22:01.640584+0000","last_clean":"2023-10-06T11:22:01.640584+0000","last_became_active":"2023-10-06T11:22:01.639253+0000","last_became_peered":"2023-10-06T11:22:01.639253+0000","last_unstale":"2023-10-06T11:22:01.640584+0000","last_undegraded":"2023-10-06T11:22:01.640584+0000","last_fullsized":"2023-10-06T11:22:01.640584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615560+0000","last_change":"2023-10-06T11:21:56.574525+0000","last_active":"2023-10-06T11:22:01.615560+0000","last_peered":"2023-10-06T11:22:01.615560+0000","last_clean":"2023-10-06T11:22:01.615560+0000","last_became_active":"2023-10-06T11:21:56.574182+0000","last_became_peered":"2023-10-06T11:21:56.574182+0000","last_unstale":"2023-10-06T11:22:01.615560+0000","last_undegraded":"2023-10-06T11:22:01.615560+0000","last_fullsized":"2023-10-06T11:22:01.615560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620100+0000","last_change":"2023-10-06T11:21:57.564759+0000","last_active":"2023-10-06T11:22:01.620100+0000","last_peered":"2023-10-06T11:22:01.620100+0000","last_clean":"2023-10-06T11:22:01.620100+0000","last_became_active":"2023-10-06T11:21:57.564385+0000","last_became_peered":"2023-10-06T11:21:57.564385+0000","last_unstale":"2023-10-06T11:22:01.620100+0000","last_undegraded":"2023-10-06T11:22:01.620100+0000","last_fullsized":"2023-10-06T11:22:01.620100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615364+0000","last_change":"2023-10-06T11:21:57.567825+0000","last_active":"2023-10-06T11:22:01.615364+0000","last_peered":"2023-10-06T11:22:01.615364+0000","last_clean":"2023-10-06T11:22:01.615364+0000","last_became_active":"2023-10-06T11:21:57.562342+0000","last_became_peered":"2023-10-06T11:21:57.562342+0000","last_unstale":"2023-10-06T11:22:01.615364+0000","last_undegraded":"2023-10-06T11:22:01.615364+0000","last_fullsized":"2023-10-06T11:22:01.615364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627753+0000","last_change":"2023-10-06T11:21:56.573919+0000","last_active":"2023-10-06T11:22:01.627753+0000","last_peered":"2023-10-06T11:22:01.627753+0000","last_clean":"2023-10-06T11:22:01.627753+0000","last_became_active":"2023-10-06T11:21:56.573605+0000","last_became_peered":"2023-10-06T11:21:56.573605+0000","last_unstale":"2023-10-06T11:22:01.627753+0000","last_undegraded":"2023-10-06T11:22:01.627753+0000","last_fullsized":"2023-10-06T11:22:01.627753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639931+0000","last_change":"2023-10-06T11:22:01.637440+0000","last_active":"2023-10-06T11:22:01.639931+0000","last_peered":"2023-10-06T11:22:01.639931+0000","last_clean":"2023-10-06T11:22:01.639931+0000","last_became_active":"2023-10-06T11:22:01.637003+0000","last_became_peered":"2023-10-06T11:22:01.637003+0000","last_unstale":"2023-10-06T11:22:01.639931+0000","last_undegraded":"2023-10-06T11:22:01.639931+0000","last_fullsized":"2023-10-06T11:22:01.639931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.578282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617665+0000","last_change":"2023-10-06T11:21:57.572470+0000","last_active":"2023-10-06T11:22:01.617665+0000","last_peered":"2023-10-06T11:22:01.617665+0000","last_clean":"2023-10-06T11:22:01.617665+0000","last_became_active":"2023-10-06T11:21:57.568699+0000","last_became_peered":"2023-10-06T11:21:57.568699+0000","last_unstale":"2023-10-06T11:22:01.617665+0000","last_undegraded":"2023-10-06T11:22:01.617665+0000","last_fullsized":"2023-10-06T11:22:01.617665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621640+0000","last_change":"2023-10-06T11:21:56.576266+0000","last_active":"2023-10-06T11:22:01.621640+0000","last_peered":"2023-10-06T11:22:01.621640+0000","last_clean":"2023-10-06T11:22:01.621640+0000","last_became_active":"2023-10-06T11:21:56.575879+0000","last_became_peered":"2023-10-06T11:21:56.575879+0000","last_unstale":"2023-10-06T11:22:01.621640+0000","last_undegraded":"2023-10-06T11:22:01.621640+0000","last_fullsized":"2023-10-06T11:22:01.621640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645265+0000","last_change":"2023-10-06T11:22:01.642796+0000","last_active":"2023-10-06T11:22:01.645265+0000","last_peered":"2023-10-06T11:22:01.645265+0000","last_clean":"2023-10-06T11:22:01.645265+0000","last_became_active":"2023-10-06T11:22:01.642102+0000","last_became_peered":"2023-10-06T11:22:01.642102+0000","last_unstale":"2023-10-06T11:22:01.645265+0000","last_undegraded":"2023-10-06T11:22:01.645265+0000","last_fullsized":"2023-10-06T11:22:01.645265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620311+0000","last_change":"2023-10-06T11:21:56.577336+0000","last_active":"2023-10-06T11:22:01.620311+0000","last_peered":"2023-10-06T11:22:01.620311+0000","last_clean":"2023-10-06T11:22:01.620311+0000","last_became_active":"2023-10-06T11:21:56.574934+0000","last_became_peered":"2023-10-06T11:21:56.574934+0000","last_unstale":"2023-10-06T11:22:01.620311+0000","last_undegraded":"2023-10-06T11:22:01.620311+0000","last_fullsized":"2023-10-06T11:22:01.620311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:58.978806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615123+0000","last_change":"2023-10-06T11:21:57.565736+0000","last_active":"2023-10-06T11:22:01.615123+0000","last_peered":"2023-10-06T11:22:01.615123+0000","last_clean":"2023-10-06T11:22:01.615123+0000","last_became_active":"2023-10-06T11:21:57.565078+0000","last_became_peered":"2023-10-06T11:21:57.565078+0000","last_unstale":"2023-10-06T11:22:01.615123+0000","last_undegraded":"2023-10-06T11:22:01.615123+0000","last_fullsized":"2023-10-06T11:22:01.615123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637186+0000","last_change":"2023-10-06T11:22:01.635715+0000","last_active":"2023-10-06T11:22:01.637186+0000","last_peered":"2023-10-06T11:22:01.637186+0000","last_clean":"2023-10-06T11:22:01.637186+0000","last_became_active":"2023-10-06T11:22:01.635213+0000","last_became_peered":"2023-10-06T11:22:01.635213+0000","last_unstale":"2023-10-06T11:22:01.637186+0000","last_undegraded":"2023-10-06T11:22:01.637186+0000","last_fullsized":"2023-10-06T11:22:01.637186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:00.176987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-06T11:22:07.460343+0000","last_change":"2023-10-06T11:21:56.573406+0000","last_active":"2023-10-06T11:22:07.460343+0000","last_peered":"2023-10-06T11:22:07.460343+0000","last_clean":"2023-10-06T11:22:07.460343+0000","last_became_active":"2023-10-06T11:21:56.572858+0000","last_became_peered":"2023-10-06T11:21:56.572858+0000","last_unstale":"2023-10-06T11:22:07.460343+0000","last_undegraded":"2023-10-06T11:22:07.460343+0000","last_fullsized":"2023-10-06T11:22:07.460343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611768+0000","last_change":"2023-10-06T11:21:57.566510+0000","last_active":"2023-10-06T11:22:01.611768+0000","last_peered":"2023-10-06T11:22:01.611768+0000","last_clean":"2023-10-06T11:22:01.611768+0000","last_became_active":"2023-10-06T11:21:57.566168+0000","last_became_peered":"2023-10-06T11:21:57.566168+0000","last_unstale":"2023-10-06T11:22:01.611768+0000","last_undegraded":"2023-10-06T11:22:01.611768+0000","last_fullsized":"2023-10-06T11:22:01.611768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:03.939980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644970+0000","last_change":"2023-10-06T11:22:01.640707+0000","last_active":"2023-10-06T11:22:01.644970+0000","last_peered":"2023-10-06T11:22:01.644970+0000","last_clean":"2023-10-06T11:22:01.644970+0000","last_became_active":"2023-10-06T11:22:01.640294+0000","last_became_peered":"2023-10-06T11:22:01.640294+0000","last_unstale":"2023-10-06T11:22:01.644970+0000","last_undegraded":"2023-10-06T11:22:01.644970+0000","last_fullsized":"2023-10-06T11:22:01.644970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:07.995705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615237+0000","last_change":"2023-10-06T11:21:57.565357+0000","last_active":"2023-10-06T11:22:01.615237+0000","last_peered":"2023-10-06T11:22:01.615237+0000","last_clean":"2023-10-06T11:22:01.615237+0000","last_became_active":"2023-10-06T11:21:57.560887+0000","last_became_peered":"2023-10-06T11:21:57.560887+0000","last_unstale":"2023-10-06T11:22:01.615237+0000","last_undegraded":"2023-10-06T11:22:01.615237+0000","last_fullsized":"2023-10-06T11:22:01.615237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620136+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:22:01.620136+0000","last_peered":"2023-10-06T11:22:01.620136+0000","last_clean":"2023-10-06T11:22:01.620136+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:22:01.620136+0000","last_undegraded":"2023-10-06T11:22:01.620136+0000","last_fullsized":"2023-10-06T11:22:01.620136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621725+0000","last_change":"2023-10-06T11:21:56.569726+0000","last_active":"2023-10-06T11:22:01.621725+0000","last_peered":"2023-10-06T11:22:01.621725+0000","last_clean":"2023-10-06T11:22:01.621725+0000","last_became_active":"2023-10-06T11:21:56.569394+0000","last_became_peered":"2023-10-06T11:21:56.569394+0000","last_unstale":"2023-10-06T11:22:01.621725+0000","last_undegraded":"2023-10-06T11:22:01.621725+0000","last_fullsized":"2023-10-06T11:22:01.621725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640548+0000","last_change":"2023-10-06T11:22:01.637613+0000","last_active":"2023-10-06T11:22:01.640548+0000","last_peered":"2023-10-06T11:22:01.640548+0000","last_clean":"2023-10-06T11:22:01.640548+0000","last_became_active":"2023-10-06T11:22:01.636818+0000","last_became_peered":"2023-10-06T11:22:01.636818+0000","last_unstale":"2023-10-06T11:22:01.640548+0000","last_undegraded":"2023-10-06T11:22:01.640548+0000","last_fullsized":"2023-10-06T11:22:01.640548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626583+0000","last_change":"2023-10-06T11:21:57.556132+0000","last_active":"2023-10-06T11:22:01.626583+0000","last_peered":"2023-10-06T11:22:01.626583+0000","last_clean":"2023-10-06T11:22:01.626583+0000","last_became_active":"2023-10-06T11:21:57.555766+0000","last_became_peered":"2023-10-06T11:21:57.555766+0000","last_unstale":"2023-10-06T11:22:01.626583+0000","last_undegraded":"2023-10-06T11:22:01.626583+0000","last_fullsized":"2023-10-06T11:22:01.626583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620468+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:22:01.620468+0000","last_peered":"2023-10-06T11:22:01.620468+0000","last_clean":"2023-10-06T11:22:01.620468+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:22:01.620468+0000","last_undegraded":"2023-10-06T11:22:01.620468+0000","last_fullsized":"2023-10-06T11:22:01.620468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619168+0000","last_change":"2023-10-06T11:21:56.570891+0000","last_active":"2023-10-06T11:22:01.619168+0000","last_peered":"2023-10-06T11:22:01.619168+0000","last_clean":"2023-10-06T11:22:01.619168+0000","last_became_active":"2023-10-06T11:21:56.570569+0000","last_became_peered":"2023-10-06T11:21:56.570569+0000","last_unstale":"2023-10-06T11:22:01.619168+0000","last_undegraded":"2023-10-06T11:22:01.619168+0000","last_fullsized":"2023-10-06T11:22:01.619168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634770+0000","last_change":"2023-10-06T11:22:01.630420+0000","last_active":"2023-10-06T11:22:01.634770+0000","last_peered":"2023-10-06T11:22:01.634770+0000","last_clean":"2023-10-06T11:22:01.634770+0000","last_became_active":"2023-10-06T11:22:01.629842+0000","last_became_peered":"2023-10-06T11:22:01.629842+0000","last_unstale":"2023-10-06T11:22:01.634770+0000","last_undegraded":"2023-10-06T11:22:01.634770+0000","last_fullsized":"2023-10-06T11:22:01.634770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:49.797072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615458+0000","last_change":"2023-10-06T11:21:57.568394+0000","last_active":"2023-10-06T11:22:01.615458+0000","last_peered":"2023-10-06T11:22:01.615458+0000","last_clean":"2023-10-06T11:22:01.615458+0000","last_became_active":"2023-10-06T11:21:57.568120+0000","last_became_peered":"2023-10-06T11:21:57.568120+0000","last_unstale":"2023-10-06T11:22:01.615458+0000","last_undegraded":"2023-10-06T11:22:01.615458+0000","last_fullsized":"2023-10-06T11:22:01.615458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610919+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:22:01.610919+0000","last_peered":"2023-10-06T11:22:01.610919+0000","last_clean":"2023-10-06T11:22:01.610919+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:22:01.610919+0000","last_undegraded":"2023-10-06T11:22:01.610919+0000","last_fullsized":"2023-10-06T11:22:01.610919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616279+0000","last_change":"2023-10-06T11:21:56.574787+0000","last_active":"2023-10-06T11:22:01.616279+0000","last_peered":"2023-10-06T11:22:01.616279+0000","last_clean":"2023-10-06T11:22:01.616279+0000","last_became_active":"2023-10-06T11:21:56.574471+0000","last_became_peered":"2023-10-06T11:21:56.574471+0000","last_unstale":"2023-10-06T11:22:01.616279+0000","last_undegraded":"2023-10-06T11:22:01.616279+0000","last_fullsized":"2023-10-06T11:22:01.616279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642714+0000","last_change":"2023-10-06T11:22:01.632835+0000","last_active":"2023-10-06T11:22:01.642714+0000","last_peered":"2023-10-06T11:22:01.642714+0000","last_clean":"2023-10-06T11:22:01.642714+0000","last_became_active":"2023-10-06T11:22:01.632391+0000","last_became_peered":"2023-10-06T11:22:01.632391+0000","last_unstale":"2023-10-06T11:22:01.642714+0000","last_undegraded":"2023-10-06T11:22:01.642714+0000","last_fullsized":"2023-10-06T11:22:01.642714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:08.263854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611307+0000","last_change":"2023-10-06T11:21:57.561918+0000","last_active":"2023-10-06T11:22:01.611307+0000","last_peered":"2023-10-06T11:22:01.611307+0000","last_clean":"2023-10-06T11:22:01.611307+0000","last_became_active":"2023-10-06T11:21:57.560917+0000","last_became_peered":"2023-10-06T11:21:57.560917+0000","last_unstale":"2023-10-06T11:22:01.611307+0000","last_undegraded":"2023-10-06T11:22:01.611307+0000","last_fullsized":"2023-10-06T11:22:01.611307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616596+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:22:01.616596+0000","last_peered":"2023-10-06T11:22:01.616596+0000","last_clean":"2023-10-06T11:22:01.616596+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:22:01.616596+0000","last_undegraded":"2023-10-06T11:22:01.616596+0000","last_fullsized":"2023-10-06T11:22:01.616596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:02.262049+0000","last_change":"2023-10-06T11:21:56.573826+0000","last_active":"2023-10-06T11:22:02.262049+0000","last_peered":"2023-10-06T11:22:02.262049+0000","last_clean":"2023-10-06T11:22:02.262049+0000","last_became_active":"2023-10-06T11:21:56.573509+0000","last_became_peered":"2023-10-06T11:21:56.573509+0000","last_unstale":"2023-10-06T11:22:02.262049+0000","last_undegraded":"2023-10-06T11:22:02.262049+0000","last_fullsized":"2023-10-06T11:22:02.262049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634403+0000","last_change":"2023-10-06T11:22:01.631312+0000","last_active":"2023-10-06T11:22:01.634403+0000","last_peered":"2023-10-06T11:22:01.634403+0000","last_clean":"2023-10-06T11:22:01.634403+0000","last_became_active":"2023-10-06T11:22:01.630877+0000","last_became_peered":"2023-10-06T11:22:01.630877+0000","last_unstale":"2023-10-06T11:22:01.634403+0000","last_undegraded":"2023-10-06T11:22:01.634403+0000","last_fullsized":"2023-10-06T11:22:01.634403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:03:28.590751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627610+0000","last_change":"2023-10-06T11:21:57.567907+0000","last_active":"2023-10-06T11:22:01.627610+0000","last_peered":"2023-10-06T11:22:01.627610+0000","last_clean":"2023-10-06T11:22:01.627610+0000","last_became_active":"2023-10-06T11:21:57.567544+0000","last_became_peered":"2023-10-06T11:21:57.567544+0000","last_unstale":"2023-10-06T11:22:01.627610+0000","last_undegraded":"2023-10-06T11:22:01.627610+0000","last_fullsized":"2023-10-06T11:22:01.627610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.625024+0000","last_change":"2023-10-06T11:21:56.571499+0000","last_active":"2023-10-06T11:22:01.625024+0000","last_peered":"2023-10-06T11:22:01.625024+0000","last_clean":"2023-10-06T11:22:01.625024+0000","last_became_active":"2023-10-06T11:21:56.571176+0000","last_became_peered":"2023-10-06T11:21:56.571176+0000","last_unstale":"2023-10-06T11:22:01.625024+0000","last_undegraded":"2023-10-06T11:22:01.625024+0000","last_fullsized":"2023-10-06T11:22:01.625024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616847+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:22:01.616847+0000","last_peered":"2023-10-06T11:22:01.616847+0000","last_clean":"2023-10-06T11:22:01.616847+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:22:01.616847+0000","last_undegraded":"2023-10-06T11:22:01.616847+0000","last_fullsized":"2023-10-06T11:22:01.616847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614070+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:22:01.614070+0000","last_peered":"2023-10-06T11:22:01.614070+0000","last_clean":"2023-10-06T11:22:01.614070+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:22:01.614070+0000","last_undegraded":"2023-10-06T11:22:01.614070+0000","last_fullsized":"2023-10-06T11:22:01.614070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635892+0000","last_change":"2023-10-06T11:22:01.632615+0000","last_active":"2023-10-06T11:22:01.635892+0000","last_peered":"2023-10-06T11:22:01.635892+0000","last_clean":"2023-10-06T11:22:01.635892+0000","last_became_active":"2023-10-06T11:22:01.632167+0000","last_became_peered":"2023-10-06T11:22:01.632167+0000","last_unstale":"2023-10-06T11:22:01.635892+0000","last_undegraded":"2023-10-06T11:22:01.635892+0000","last_fullsized":"2023-10-06T11:22:01.635892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:35.069167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614553+0000","last_change":"2023-10-06T11:21:57.558911+0000","last_active":"2023-10-06T11:22:01.614553+0000","last_peered":"2023-10-06T11:22:01.614553+0000","last_clean":"2023-10-06T11:22:01.614553+0000","last_became_active":"2023-10-06T11:21:57.555525+0000","last_became_peered":"2023-10-06T11:21:57.555525+0000","last_unstale":"2023-10-06T11:22:01.614553+0000","last_undegraded":"2023-10-06T11:22:01.614553+0000","last_fullsized":"2023-10-06T11:22:01.614553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620129+0000","last_change":"2023-10-06T11:21:56.566441+0000","last_active":"2023-10-06T11:22:01.620129+0000","last_peered":"2023-10-06T11:22:01.620129+0000","last_clean":"2023-10-06T11:22:01.620129+0000","last_became_active":"2023-10-06T11:21:56.566070+0000","last_became_peered":"2023-10-06T11:21:56.566070+0000","last_unstale":"2023-10-06T11:22:01.620129+0000","last_undegraded":"2023-10-06T11:22:01.620129+0000","last_fullsized":"2023-10-06T11:22:01.620129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616748+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:22:01.616748+0000","last_peered":"2023-10-06T11:22:01.616748+0000","last_clean":"2023-10-06T11:22:01.616748+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:22:01.616748+0000","last_undegraded":"2023-10-06T11:22:01.616748+0000","last_fullsized":"2023-10-06T11:22:01.616748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637816+0000","last_change":"2023-10-06T11:22:01.632881+0000","last_active":"2023-10-06T11:22:01.637816+0000","last_peered":"2023-10-06T11:22:01.637816+0000","last_clean":"2023-10-06T11:22:01.637816+0000","last_became_active":"2023-10-06T11:22:01.632330+0000","last_became_peered":"2023-10-06T11:22:01.632330+0000","last_unstale":"2023-10-06T11:22:01.637816+0000","last_undegraded":"2023-10-06T11:22:01.637816+0000","last_fullsized":"2023-10-06T11:22:01.637816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:38.278034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619088+0000","last_change":"2023-10-06T11:21:57.565393+0000","last_active":"2023-10-06T11:22:01.619088+0000","last_peered":"2023-10-06T11:22:01.619088+0000","last_clean":"2023-10-06T11:22:01.619088+0000","last_became_active":"2023-10-06T11:21:57.565128+0000","last_became_peered":"2023-10-06T11:21:57.565128+0000","last_unstale":"2023-10-06T11:22:01.619088+0000","last_undegraded":"2023-10-06T11:22:01.619088+0000","last_fullsized":"2023-10-06T11:22:01.619088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611906+0000","last_change":"2023-10-06T11:21:56.573109+0000","last_active":"2023-10-06T11:22:01.611906+0000","last_peered":"2023-10-06T11:22:01.611906+0000","last_clean":"2023-10-06T11:22:01.611906+0000","last_became_active":"2023-10-06T11:21:56.572783+0000","last_became_peered":"2023-10-06T11:21:56.572783+0000","last_unstale":"2023-10-06T11:22:01.611906+0000","last_undegraded":"2023-10-06T11:22:01.611906+0000","last_fullsized":"2023-10-06T11:22:01.611906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620654+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:22:01.620654+0000","last_peered":"2023-10-06T11:22:01.620654+0000","last_clean":"2023-10-06T11:22:01.620654+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:22:01.620654+0000","last_undegraded":"2023-10-06T11:22:01.620654+0000","last_fullsized":"2023-10-06T11:22:01.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643525+0000","last_change":"2023-10-06T11:22:01.641973+0000","last_active":"2023-10-06T11:22:01.643525+0000","last_peered":"2023-10-06T11:22:01.643525+0000","last_clean":"2023-10-06T11:22:01.643525+0000","last_became_active":"2023-10-06T11:22:01.641394+0000","last_became_peered":"2023-10-06T11:22:01.641394+0000","last_unstale":"2023-10-06T11:22:01.643525+0000","last_undegraded":"2023-10-06T11:22:01.643525+0000","last_fullsized":"2023-10-06T11:22:01.643525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620183+0000","last_change":"2023-10-06T11:21:57.564937+0000","last_active":"2023-10-06T11:22:01.620183+0000","last_peered":"2023-10-06T11:22:01.620183+0000","last_clean":"2023-10-06T11:22:01.620183+0000","last_became_active":"2023-10-06T11:21:57.564603+0000","last_became_peered":"2023-10-06T11:21:57.564603+0000","last_unstale":"2023-10-06T11:22:01.620183+0000","last_undegraded":"2023-10-06T11:22:01.620183+0000","last_fullsized":"2023-10-06T11:22:01.620183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615864+0000","last_change":"2023-10-06T11:21:56.567945+0000","last_active":"2023-10-06T11:22:01.615864+0000","last_peered":"2023-10-06T11:22:01.615864+0000","last_clean":"2023-10-06T11:22:01.615864+0000","last_became_active":"2023-10-06T11:21:56.567592+0000","last_became_peered":"2023-10-06T11:21:56.567592+0000","last_unstale":"2023-10-06T11:22:01.615864+0000","last_undegraded":"2023-10-06T11:22:01.615864+0000","last_fullsized":"2023-10-06T11:22:01.615864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616944+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:22:01.616944+0000","last_peered":"2023-10-06T11:22:01.616944+0000","last_clean":"2023-10-06T11:22:01.616944+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:22:01.616944+0000","last_undegraded":"2023-10-06T11:22:01.616944+0000","last_fullsized":"2023-10-06T11:22:01.616944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644814+0000","last_change":"2023-10-06T11:22:01.639193+0000","last_active":"2023-10-06T11:22:01.644814+0000","last_peered":"2023-10-06T11:22:01.644814+0000","last_clean":"2023-10-06T11:22:01.644814+0000","last_became_active":"2023-10-06T11:22:01.638730+0000","last_became_peered":"2023-10-06T11:22:01.638730+0000","last_unstale":"2023-10-06T11:22:01.644814+0000","last_undegraded":"2023-10-06T11:22:01.644814+0000","last_fullsized":"2023-10-06T11:22:01.644814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614448+0000","last_change":"2023-10-06T11:21:57.565173+0000","last_active":"2023-10-06T11:22:01.614448+0000","last_peered":"2023-10-06T11:22:01.614448+0000","last_clean":"2023-10-06T11:22:01.614448+0000","last_became_active":"2023-10-06T11:21:57.564865+0000","last_became_peered":"2023-10-06T11:21:57.564865+0000","last_unstale":"2023-10-06T11:22:01.614448+0000","last_undegraded":"2023-10-06T11:22:01.614448+0000","last_fullsized":"2023-10-06T11:22:01.614448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616901+0000","last_change":"2023-10-06T11:21:56.570015+0000","last_active":"2023-10-06T11:22:01.616901+0000","last_peered":"2023-10-06T11:22:01.616901+0000","last_clean":"2023-10-06T11:22:01.616901+0000","last_became_active":"2023-10-06T11:21:56.569530+0000","last_became_peered":"2023-10-06T11:21:56.569530+0000","last_unstale":"2023-10-06T11:22:01.616901+0000","last_undegraded":"2023-10-06T11:22:01.616901+0000","last_fullsized":"2023-10-06T11:22:01.616901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642672+0000","last_change":"2023-10-06T11:22:01.632016+0000","last_active":"2023-10-06T11:22:01.642672+0000","last_peered":"2023-10-06T11:22:01.642672+0000","last_clean":"2023-10-06T11:22:01.642672+0000","last_became_active":"2023-10-06T11:22:01.629544+0000","last_became_peered":"2023-10-06T11:22:01.629544+0000","last_unstale":"2023-10-06T11:22:01.642672+0000","last_undegraded":"2023-10-06T11:22:01.642672+0000","last_fullsized":"2023-10-06T11:22:01.642672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:20.535717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611336+0000","last_change":"2023-10-06T11:21:57.566827+0000","last_active":"2023-10-06T11:22:01.611336+0000","last_peered":"2023-10-06T11:22:01.611336+0000","last_clean":"2023-10-06T11:22:01.611336+0000","last_became_active":"2023-10-06T11:21:57.566463+0000","last_became_peered":"2023-10-06T11:21:57.566463+0000","last_unstale":"2023-10-06T11:22:01.611336+0000","last_undegraded":"2023-10-06T11:22:01.611336+0000","last_fullsized":"2023-10-06T11:22:01.611336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611609+0000","last_change":"2023-10-06T11:21:56.573082+0000","last_active":"2023-10-06T11:22:01.611609+0000","last_peered":"2023-10-06T11:22:01.611609+0000","last_clean":"2023-10-06T11:22:01.611609+0000","last_became_active":"2023-10-06T11:21:56.572363+0000","last_became_peered":"2023-10-06T11:21:56.572363+0000","last_unstale":"2023-10-06T11:22:01.611609+0000","last_undegraded":"2023-10-06T11:22:01.611609+0000","last_fullsized":"2023-10-06T11:22:01.611609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.628822+0000","last_change":"2023-10-06T11:22:01.626277+0000","last_active":"2023-10-06T11:22:01.628822+0000","last_peered":"2023-10-06T11:22:01.628822+0000","last_clean":"2023-10-06T11:22:01.628822+0000","last_became_active":"2023-10-06T11:22:01.625806+0000","last_became_peered":"2023-10-06T11:22:01.625806+0000","last_unstale":"2023-10-06T11:22:01.628822+0000","last_undegraded":"2023-10-06T11:22:01.628822+0000","last_fullsized":"2023-10-06T11:22:01.628822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611575+0000","last_change":"2023-10-06T11:21:57.564883+0000","last_active":"2023-10-06T11:22:01.611575+0000","last_peered":"2023-10-06T11:22:01.611575+0000","last_clean":"2023-10-06T11:22:01.611575+0000","last_became_active":"2023-10-06T11:21:57.564586+0000","last_became_peered":"2023-10-06T11:21:57.564586+0000","last_unstale":"2023-10-06T11:22:01.611575+0000","last_undegraded":"2023-10-06T11:22:01.611575+0000","last_fullsized":"2023-10-06T11:22:01.611575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:12.024410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621041+0000","last_change":"2023-10-06T11:21:56.576790+0000","last_active":"2023-10-06T11:22:01.621041+0000","last_peered":"2023-10-06T11:22:01.621041+0000","last_clean":"2023-10-06T11:22:01.621041+0000","last_became_active":"2023-10-06T11:21:56.576463+0000","last_became_peered":"2023-10-06T11:21:56.576463+0000","last_unstale":"2023-10-06T11:22:01.621041+0000","last_undegraded":"2023-10-06T11:22:01.621041+0000","last_fullsized":"2023-10-06T11:22:01.621041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643771+0000","last_change":"2023-10-06T11:22:01.639619+0000","last_active":"2023-10-06T11:22:01.643771+0000","last_peered":"2023-10-06T11:22:01.643771+0000","last_clean":"2023-10-06T11:22:01.643771+0000","last_became_active":"2023-10-06T11:22:01.638615+0000","last_became_peered":"2023-10-06T11:22:01.638615+0000","last_unstale":"2023-10-06T11:22:01.643771+0000","last_undegraded":"2023-10-06T11:22:01.643771+0000","last_fullsized":"2023-10-06T11:22:01.643771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612709+0000","last_change":"2023-10-06T11:21:57.577150+0000","last_active":"2023-10-06T11:22:01.612709+0000","last_peered":"2023-10-06T11:22:01.612709+0000","last_clean":"2023-10-06T11:22:01.612709+0000","last_became_active":"2023-10-06T11:21:57.576632+0000","last_became_peered":"2023-10-06T11:21:57.576632+0000","last_unstale":"2023-10-06T11:22:01.612709+0000","last_undegraded":"2023-10-06T11:22:01.612709+0000","last_fullsized":"2023-10-06T11:22:01.612709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:32.416966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617372+0000","last_change":"2023-10-06T11:21:56.575818+0000","last_active":"2023-10-06T11:22:01.617372+0000","last_peered":"2023-10-06T11:22:01.617372+0000","last_clean":"2023-10-06T11:22:01.617372+0000","last_became_active":"2023-10-06T11:21:56.575460+0000","last_became_peered":"2023-10-06T11:21:56.575460+0000","last_unstale":"2023-10-06T11:22:01.617372+0000","last_undegraded":"2023-10-06T11:22:01.617372+0000","last_fullsized":"2023-10-06T11:22:01.617372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634446+0000","last_change":"2023-10-06T11:22:01.632266+0000","last_active":"2023-10-06T11:22:01.634446+0000","last_peered":"2023-10-06T11:22:01.634446+0000","last_clean":"2023-10-06T11:22:01.634446+0000","last_became_active":"2023-10-06T11:22:01.631851+0000","last_became_peered":"2023-10-06T11:22:01.631851+0000","last_unstale":"2023-10-06T11:22:01.634446+0000","last_undegraded":"2023-10-06T11:22:01.634446+0000","last_fullsized":"2023-10-06T11:22:01.634446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:11.811755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626480+0000","last_change":"2023-10-06T11:21:57.561599+0000","last_active":"2023-10-06T11:22:01.626480+0000","last_peered":"2023-10-06T11:22:01.626480+0000","last_clean":"2023-10-06T11:22:01.626480+0000","last_became_active":"2023-10-06T11:21:57.561280+0000","last_became_peered":"2023-10-06T11:21:57.561280+0000","last_unstale":"2023-10-06T11:22:01.626480+0000","last_undegraded":"2023-10-06T11:22:01.626480+0000","last_fullsized":"2023-10-06T11:22:01.626480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621119+0000","last_change":"2023-10-06T11:21:56.566817+0000","last_active":"2023-10-06T11:22:01.621119+0000","last_peered":"2023-10-06T11:22:01.621119+0000","last_clean":"2023-10-06T11:22:01.621119+0000","last_became_active":"2023-10-06T11:21:56.566466+0000","last_became_peered":"2023-10-06T11:21:56.566466+0000","last_unstale":"2023-10-06T11:22:01.621119+0000","last_undegraded":"2023-10-06T11:22:01.621119+0000","last_fullsized":"2023-10-06T11:22:01.621119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.632874+0000","last_change":"2023-10-06T11:22:01.628287+0000","last_active":"2023-10-06T11:22:01.632874+0000","last_peered":"2023-10-06T11:22:01.632874+0000","last_clean":"2023-10-06T11:22:01.632874+0000","last_became_active":"2023-10-06T11:22:01.627842+0000","last_became_peered":"2023-10-06T11:22:01.627842+0000","last_unstale":"2023-10-06T11:22:01.632874+0000","last_undegraded":"2023-10-06T11:22:01.632874+0000","last_fullsized":"2023-10-06T11:22:01.632874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:42.027969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620876+0000","last_change":"2023-10-06T11:21:57.572846+0000","last_active":"2023-10-06T11:22:01.620876+0000","last_peered":"2023-10-06T11:22:01.620876+0000","last_clean":"2023-10-06T11:22:01.620876+0000","last_became_active":"2023-10-06T11:21:57.572567+0000","last_became_peered":"2023-10-06T11:21:57.572567+0000","last_unstale":"2023-10-06T11:22:01.620876+0000","last_undegraded":"2023-10-06T11:22:01.620876+0000","last_fullsized":"2023-10-06T11:22:01.620876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620369+0000","last_change":"2023-10-06T11:21:56.575220+0000","last_active":"2023-10-06T11:22:01.620369+0000","last_peered":"2023-10-06T11:22:01.620369+0000","last_clean":"2023-10-06T11:22:01.620369+0000","last_became_active":"2023-10-06T11:21:56.574557+0000","last_became_peered":"2023-10-06T11:21:56.574557+0000","last_unstale":"2023-10-06T11:22:01.620369+0000","last_undegraded":"2023-10-06T11:22:01.620369+0000","last_fullsized":"2023-10-06T11:22:01.620369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639894+0000","last_change":"2023-10-06T11:22:01.637484+0000","last_active":"2023-10-06T11:22:01.639894+0000","last_peered":"2023-10-06T11:22:01.639894+0000","last_clean":"2023-10-06T11:22:01.639894+0000","last_became_active":"2023-10-06T11:22:01.637045+0000","last_became_peered":"2023-10-06T11:22:01.637045+0000","last_unstale":"2023-10-06T11:22:01.639894+0000","last_undegraded":"2023-10-06T11:22:01.639894+0000","last_fullsized":"2023-10-06T11:22:01.639894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:36.472691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610908+0000","last_change":"2023-10-06T11:21:57.562666+0000","last_active":"2023-10-06T11:22:01.610908+0000","last_peered":"2023-10-06T11:22:01.610908+0000","last_clean":"2023-10-06T11:22:01.610908+0000","last_became_active":"2023-10-06T11:21:57.562384+0000","last_became_peered":"2023-10-06T11:21:57.562384+0000","last_unstale":"2023-10-06T11:22:01.610908+0000","last_undegraded":"2023-10-06T11:22:01.610908+0000","last_fullsized":"2023-10-06T11:22:01.610908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:03.832047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619654+0000","last_change":"2023-10-06T11:21:56.578885+0000","last_active":"2023-10-06T11:22:01.619654+0000","last_peered":"2023-10-06T11:22:01.619654+0000","last_clean":"2023-10-06T11:22:01.619654+0000","last_became_active":"2023-10-06T11:21:56.578589+0000","last_became_peered":"2023-10-06T11:21:56.578589+0000","last_unstale":"2023-10-06T11:22:01.619654+0000","last_undegraded":"2023-10-06T11:22:01.619654+0000","last_fullsized":"2023-10-06T11:22:01.619654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:31.263377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.629203+0000","last_change":"2023-10-06T11:22:01.626674+0000","last_active":"2023-10-06T11:22:01.629203+0000","last_peered":"2023-10-06T11:22:01.629203+0000","last_clean":"2023-10-06T11:22:01.629203+0000","last_became_active":"2023-10-06T11:22:01.626199+0000","last_became_peered":"2023-10-06T11:22:01.626199+0000","last_unstale":"2023-10-06T11:22:01.629203+0000","last_undegraded":"2023-10-06T11:22:01.629203+0000","last_fullsized":"2023-10-06T11:22:01.629203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:07.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619746+0000","last_change":"2023-10-06T11:21:57.568271+0000","last_active":"2023-10-06T11:22:01.619746+0000","last_peered":"2023-10-06T11:22:01.619746+0000","last_clean":"2023-10-06T11:22:01.619746+0000","last_became_active":"2023-10-06T11:21:57.567943+0000","last_became_peered":"2023-10-06T11:21:57.567943+0000","last_unstale":"2023-10-06T11:22:01.619746+0000","last_undegraded":"2023-10-06T11:22:01.619746+0000","last_fullsized":"2023-10-06T11:22:01.619746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619520+0000","last_change":"2023-10-06T11:21:56.579019+0000","last_active":"2023-10-06T11:22:01.619520+0000","last_peered":"2023-10-06T11:22:01.619520+0000","last_clean":"2023-10-06T11:22:01.619520+0000","last_became_active":"2023-10-06T11:21:56.578659+0000","last_became_peered":"2023-10-06T11:21:56.578659+0000","last_unstale":"2023-10-06T11:22:01.619520+0000","last_undegraded":"2023-10-06T11:22:01.619520+0000","last_fullsized":"2023-10-06T11:22:01.619520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643049+0000","last_change":"2023-10-06T11:22:01.637818+0000","last_active":"2023-10-06T11:22:01.643049+0000","last_peered":"2023-10-06T11:22:01.643049+0000","last_clean":"2023-10-06T11:22:01.643049+0000","last_became_active":"2023-10-06T11:22:01.637325+0000","last_became_peered":"2023-10-06T11:22:01.637325+0000","last_unstale":"2023-10-06T11:22:01.643049+0000","last_undegraded":"2023-10-06T11:22:01.643049+0000","last_fullsized":"2023-10-06T11:22:01.643049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641080+0000","last_change":"2023-10-06T11:22:01.636779+0000","last_active":"2023-10-06T11:22:01.641080+0000","last_peered":"2023-10-06T11:22:01.641080+0000","last_clean":"2023-10-06T11:22:01.641080+0000","last_became_active":"2023-10-06T11:22:01.636344+0000","last_became_peered":"2023-10-06T11:22:01.636344+0000","last_unstale":"2023-10-06T11:22:01.641080+0000","last_undegraded":"2023-10-06T11:22:01.641080+0000","last_fullsized":"2023-10-06T11:22:01.641080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620730+0000","last_change":"2023-10-06T11:21:56.575584+0000","last_active":"2023-10-06T11:22:01.620730+0000","last_peered":"2023-10-06T11:22:01.620730+0000","last_clean":"2023-10-06T11:22:01.620730+0000","last_became_active":"2023-10-06T11:21:56.575201+0000","last_became_peered":"2023-10-06T11:21:56.575201+0000","last_unstale":"2023-10-06T11:22:01.620730+0000","last_undegraded":"2023-10-06T11:22:01.620730+0000","last_fullsized":"2023-10-06T11:22:01.620730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616376+0000","last_change":"2023-10-06T11:21:57.568259+0000","last_active":"2023-10-06T11:22:01.616376+0000","last_peered":"2023-10-06T11:22:01.616376+0000","last_clean":"2023-10-06T11:22:01.616376+0000","last_became_active":"2023-10-06T11:21:57.567958+0000","last_became_peered":"2023-10-06T11:21:57.567958+0000","last_unstale":"2023-10-06T11:22:01.616376+0000","last_undegraded":"2023-10-06T11:22:01.616376+0000","last_fullsized":"2023-10-06T11:22:01.616376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637658+0000","last_change":"2023-10-06T11:22:01.635766+0000","last_active":"2023-10-06T11:22:01.637658+0000","last_peered":"2023-10-06T11:22:01.637658+0000","last_clean":"2023-10-06T11:22:01.637658+0000","last_became_active":"2023-10-06T11:22:01.635314+0000","last_became_peered":"2023-10-06T11:22:01.635314+0000","last_unstale":"2023-10-06T11:22:01.637658+0000","last_undegraded":"2023-10-06T11:22:01.637658+0000","last_fullsized":"2023-10-06T11:22:01.637658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:33:19.459200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619865+0000","last_change":"2023-10-06T11:21:56.579444+0000","last_active":"2023-10-06T11:22:01.619865+0000","last_peered":"2023-10-06T11:22:01.619865+0000","last_clean":"2023-10-06T11:22:01.619865+0000","last_became_active":"2023-10-06T11:21:56.578427+0000","last_became_peered":"2023-10-06T11:21:56.578427+0000","last_unstale":"2023-10-06T11:22:01.619865+0000","last_undegraded":"2023-10-06T11:22:01.619865+0000","last_fullsized":"2023-10-06T11:22:01.619865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:21.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627696+0000","last_change":"2023-10-06T11:21:57.569150+0000","last_active":"2023-10-06T11:22:01.627696+0000","last_peered":"2023-10-06T11:22:01.627696+0000","last_clean":"2023-10-06T11:22:01.627696+0000","last_became_active":"2023-10-06T11:21:57.568883+0000","last_became_peered":"2023-10-06T11:21:57.568883+0000","last_unstale":"2023-10-06T11:22:01.627696+0000","last_undegraded":"2023-10-06T11:22:01.627696+0000","last_fullsized":"2023-10-06T11:22:01.627696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626255+0000","last_change":"2023-10-06T11:21:56.572563+0000","last_active":"2023-10-06T11:22:01.626255+0000","last_peered":"2023-10-06T11:22:01.626255+0000","last_clean":"2023-10-06T11:22:01.626255+0000","last_became_active":"2023-10-06T11:21:56.571525+0000","last_became_peered":"2023-10-06T11:21:56.571525+0000","last_unstale":"2023-10-06T11:22:01.626255+0000","last_undegraded":"2023-10-06T11:22:01.626255+0000","last_fullsized":"2023-10-06T11:22:01.626255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621535+0000","last_change":"2023-10-06T11:21:57.569979+0000","last_active":"2023-10-06T11:22:01.621535+0000","last_peered":"2023-10-06T11:22:01.621535+0000","last_clean":"2023-10-06T11:22:01.621535+0000","last_became_active":"2023-10-06T11:21:57.569669+0000","last_became_peered":"2023-10-06T11:21:57.569669+0000","last_unstale":"2023-10-06T11:22:01.621535+0000","last_undegraded":"2023-10-06T11:22:01.621535+0000","last_fullsized":"2023-10-06T11:22:01.621535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.638165+0000","last_change":"2023-10-06T11:22:01.636951+0000","last_active":"2023-10-06T11:22:01.638165+0000","last_peered":"2023-10-06T11:22:01.638165+0000","last_clean":"2023-10-06T11:22:01.638165+0000","last_became_active":"2023-10-06T11:22:01.636534+0000","last_became_peered":"2023-10-06T11:22:01.636534+0000","last_unstale":"2023-10-06T11:22:01.638165+0000","last_undegraded":"2023-10-06T11:22:01.638165+0000","last_fullsized":"2023-10-06T11:22:01.638165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621855+0000","last_change":"2023-10-06T11:21:56.579219+0000","last_active":"2023-10-06T11:22:01.621855+0000","last_peered":"2023-10-06T11:22:01.621855+0000","last_clean":"2023-10-06T11:22:01.621855+0000","last_became_active":"2023-10-06T11:21:56.578100+0000","last_became_peered":"2023-10-06T11:21:56.578100+0000","last_unstale":"2023-10-06T11:22:01.621855+0000","last_undegraded":"2023-10-06T11:22:01.621855+0000","last_fullsized":"2023-10-06T11:22:01.621855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616499+0000","last_change":"2023-10-06T11:21:57.580696+0000","last_active":"2023-10-06T11:22:01.616499+0000","last_peered":"2023-10-06T11:22:01.616499+0000","last_clean":"2023-10-06T11:22:01.616499+0000","last_became_active":"2023-10-06T11:21:57.580262+0000","last_became_peered":"2023-10-06T11:21:57.580262+0000","last_unstale":"2023-10-06T11:22:01.616499+0000","last_undegraded":"2023-10-06T11:22:01.616499+0000","last_fullsized":"2023-10-06T11:22:01.616499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635928+0000","last_change":"2023-10-06T11:22:01.632977+0000","last_active":"2023-10-06T11:22:01.635928+0000","last_peered":"2023-10-06T11:22:01.635928+0000","last_clean":"2023-10-06T11:22:01.635928+0000","last_became_active":"2023-10-06T11:22:01.632522+0000","last_became_peered":"2023-10-06T11:22:01.632522+0000","last_unstale":"2023-10-06T11:22:01.635928+0000","last_undegraded":"2023-10-06T11:22:01.635928+0000","last_fullsized":"2023-10-06T11:22:01.635928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620591+0000","last_change":"2023-10-06T11:21:57.570931+0000","last_active":"2023-10-06T11:22:01.620591+0000","last_peered":"2023-10-06T11:22:01.620591+0000","last_clean":"2023-10-06T11:22:01.620591+0000","last_became_active":"2023-10-06T11:21:57.570619+0000","last_became_peered":"2023-10-06T11:21:57.570619+0000","last_unstale":"2023-10-06T11:22:01.620591+0000","last_undegraded":"2023-10-06T11:22:01.620591+0000","last_fullsized":"2023-10-06T11:22:01.620591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615615+0000","last_change":"2023-10-06T11:21:57.572767+0000","last_active":"2023-10-06T11:22:01.615615+0000","last_peered":"2023-10-06T11:22:01.615615+0000","last_clean":"2023-10-06T11:22:01.615615+0000","last_became_active":"2023-10-06T11:21:57.572371+0000","last_became_peered":"2023-10-06T11:21:57.572371+0000","last_unstale":"2023-10-06T11:22:01.615615+0000","last_undegraded":"2023-10-06T11:22:01.615615+0000","last_fullsized":"2023-10-06T11:22:01.615615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610290+0000","last_change":"2023-10-06T11:21:57.571303+0000","last_active":"2023-10-06T11:22:01.610290+0000","last_peered":"2023-10-06T11:22:01.610290+0000","last_clean":"2023-10-06T11:22:01.610290+0000","last_became_active":"2023-10-06T11:21:57.570978+0000","last_became_peered":"2023-10-06T11:21:57.570978+0000","last_unstale":"2023-10-06T11:22:01.610290+0000","last_undegraded":"2023-10-06T11:22:01.610290+0000","last_fullsized":"2023-10-06T11:22:01.610290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617222+0000","last_change":"2023-10-06T11:21:57.569031+0000","last_active":"2023-10-06T11:22:01.617222+0000","last_peered":"2023-10-06T11:22:01.617222+0000","last_clean":"2023-10-06T11:22:01.617222+0000","last_became_active":"2023-10-06T11:21:57.568734+0000","last_became_peered":"2023-10-06T11:21:57.568734+0000","last_unstale":"2023-10-06T11:22:01.617222+0000","last_undegraded":"2023-10-06T11:22:01.617222+0000","last_fullsized":"2023-10-06T11:22:01.617222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619002+0000","last_change":"2023-10-06T11:21:57.579632+0000","last_active":"2023-10-06T11:22:01.619002+0000","last_peered":"2023-10-06T11:22:01.619002+0000","last_clean":"2023-10-06T11:22:01.619002+0000","last_became_active":"2023-10-06T11:21:57.567143+0000","last_became_peered":"2023-10-06T11:21:57.567143+0000","last_unstale":"2023-10-06T11:22:01.619002+0000","last_undegraded":"2023-10-06T11:22:01.619002+0000","last_fullsized":"2023-10-06T11:22:01.619002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621442+0000","last_change":"2023-10-06T11:21:57.569847+0000","last_active":"2023-10-06T11:22:01.621442+0000","last_peered":"2023-10-06T11:22:01.621442+0000","last_clean":"2023-10-06T11:22:01.621442+0000","last_became_active":"2023-10-06T11:21:57.569571+0000","last_became_peered":"2023-10-06T11:21:57.569571+0000","last_unstale":"2023-10-06T11:22:01.621442+0000","last_undegraded":"2023-10-06T11:22:01.621442+0000","last_fullsized":"2023-10-06T11:22:01.621442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621862+0000","last_change":"2023-10-06T11:21:57.562969+0000","last_active":"2023-10-06T11:22:01.621862+0000","last_peered":"2023-10-06T11:22:01.621862+0000","last_clean":"2023-10-06T11:22:01.621862+0000","last_became_active":"2023-10-06T11:21:57.562623+0000","last_became_peered":"2023-10-06T11:21:57.562623+0000","last_unstale":"2023-10-06T11:22:01.621862+0000","last_undegraded":"2023-10-06T11:22:01.621862+0000","last_fullsized":"2023-10-06T11:22:01.621862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614335+0000","last_change":"2023-10-06T11:21:57.571639+0000","last_active":"2023-10-06T11:22:01.614335+0000","last_peered":"2023-10-06T11:22:01.614335+0000","last_clean":"2023-10-06T11:22:01.614335+0000","last_became_active":"2023-10-06T11:21:57.571347+0000","last_became_peered":"2023-10-06T11:21:57.571347+0000","last_unstale":"2023-10-06T11:22:01.614335+0000","last_undegraded":"2023-10-06T11:22:01.614335+0000","last_fullsized":"2023-10-06T11:22:01.614335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616721+0000","last_change":"2023-10-06T11:21:57.576904+0000","last_active":"2023-10-06T11:22:01.616721+0000","last_peered":"2023-10-06T11:22:01.616721+0000","last_clean":"2023-10-06T11:22:01.616721+0000","last_became_active":"2023-10-06T11:21:57.576570+0000","last_became_peered":"2023-10-06T11:21:57.576570+0000","last_unstale":"2023-10-06T11:22:01.616721+0000","last_undegraded":"2023-10-06T11:22:01.616721+0000","last_fullsized":"2023-10-06T11:22:01.616721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615508+0000","last_change":"2023-10-06T11:21:57.572666+0000","last_active":"2023-10-06T11:22:01.615508+0000","last_peered":"2023-10-06T11:22:01.615508+0000","last_clean":"2023-10-06T11:22:01.615508+0000","last_became_active":"2023-10-06T11:21:57.569109+0000","last_became_peered":"2023-10-06T11:21:57.569109+0000","last_unstale":"2023-10-06T11:22:01.615508+0000","last_undegraded":"2023-10-06T11:22:01.615508+0000","last_fullsized":"2023-10-06T11:22:01.615508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626091+0000","last_change":"2023-10-06T11:21:57.566406+0000","last_active":"2023-10-06T11:22:01.626091+0000","last_peered":"2023-10-06T11:22:01.626091+0000","last_clean":"2023-10-06T11:22:01.626091+0000","last_became_active":"2023-10-06T11:21:57.565954+0000","last_became_peered":"2023-10-06T11:21:57.565954+0000","last_unstale":"2023-10-06T11:22:01.626091+0000","last_undegraded":"2023-10-06T11:22:01.626091+0000","last_fullsized":"2023-10-06T11:22:01.626091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621753+0000","last_change":"2023-10-06T11:21:57.576914+0000","last_active":"2023-10-06T11:22:01.621753+0000","last_peered":"2023-10-06T11:22:01.621753+0000","last_clean":"2023-10-06T11:22:01.621753+0000","last_became_active":"2023-10-06T11:21:57.576444+0000","last_became_peered":"2023-10-06T11:21:57.576444+0000","last_unstale":"2023-10-06T11:22:01.621753+0000","last_undegraded":"2023-10-06T11:22:01.621753+0000","last_fullsized":"2023-10-06T11:22:01.621753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612641+0000","last_change":"2023-10-06T11:21:57.565842+0000","last_active":"2023-10-06T11:22:01.612641+0000","last_peered":"2023-10-06T11:22:01.612641+0000","last_clean":"2023-10-06T11:22:01.612641+0000","last_became_active":"2023-10-06T11:21:57.565455+0000","last_became_peered":"2023-10-06T11:21:57.565455+0000","last_unstale":"2023-10-06T11:22:01.612641+0000","last_undegraded":"2023-10-06T11:22:01.612641+0000","last_fullsized":"2023-10-06T11:22:01.612641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:28.522557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612331+0000","last_change":"2023-10-06T11:21:57.569901+0000","last_active":"2023-10-06T11:22:01.612331+0000","last_peered":"2023-10-06T11:22:01.612331+0000","last_clean":"2023-10-06T11:22:01.612331+0000","last_became_active":"2023-10-06T11:21:57.569642+0000","last_became_peered":"2023-10-06T11:21:57.569642+0000","last_unstale":"2023-10-06T11:22:01.612331+0000","last_undegraded":"2023-10-06T11:22:01.612331+0000","last_fullsized":"2023-10-06T11:22:01.612331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.950303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611118+0000","last_change":"2023-10-06T11:21:57.564154+0000","last_active":"2023-10-06T11:22:01.611118+0000","last_peered":"2023-10-06T11:22:01.611118+0000","last_clean":"2023-10-06T11:22:01.611118+0000","last_became_active":"2023-10-06T11:21:57.563840+0000","last_became_peered":"2023-10-06T11:21:57.563840+0000","last_unstale":"2023-10-06T11:22:01.611118+0000","last_undegraded":"2023-10-06T11:22:01.611118+0000","last_fullsized":"2023-10-06T11:22:01.611118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627637+0000","last_change":"2023-10-06T11:21:57.569223+0000","last_active":"2023-10-06T11:22:01.627637+0000","last_peered":"2023-10-06T11:22:01.627637+0000","last_clean":"2023-10-06T11:22:01.627637+0000","last_became_active":"2023-10-06T11:21:57.568967+0000","last_became_peered":"2023-10-06T11:21:57.568967+0000","last_unstale":"2023-10-06T11:22:01.627637+0000","last_undegraded":"2023-10-06T11:22:01.627637+0000","last_fullsized":"2023-10-06T11:22:01.627637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612585+0000","last_change":"2023-10-06T11:21:57.567578+0000","last_active":"2023-10-06T11:22:01.612585+0000","last_peered":"2023-10-06T11:22:01.612585+0000","last_clean":"2023-10-06T11:22:01.612585+0000","last_became_active":"2023-10-06T11:21:57.565977+0000","last_became_peered":"2023-10-06T11:21:57.565977+0000","last_unstale":"2023-10-06T11:22:01.612585+0000","last_undegraded":"2023-10-06T11:22:01.612585+0000","last_fullsized":"2023-10-06T11:22:01.612585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619766+0000","last_change":"2023-10-06T11:21:57.576090+0000","last_active":"2023-10-06T11:22:01.619766+0000","last_peered":"2023-10-06T11:22:01.619766+0000","last_clean":"2023-10-06T11:22:01.619766+0000","last_became_active":"2023-10-06T11:21:57.575667+0000","last_became_peered":"2023-10-06T11:21:57.575667+0000","last_unstale":"2023-10-06T11:22:01.619766+0000","last_undegraded":"2023-10-06T11:22:01.619766+0000","last_fullsized":"2023-10-06T11:22:01.619766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621178+0000","last_change":"2023-10-06T11:21:57.563777+0000","last_active":"2023-10-06T11:22:01.621178+0000","last_peered":"2023-10-06T11:22:01.621178+0000","last_clean":"2023-10-06T11:22:01.621178+0000","last_became_active":"2023-10-06T11:21:57.563485+0000","last_became_peered":"2023-10-06T11:21:57.563485+0000","last_unstale":"2023-10-06T11:22:01.621178+0000","last_undegraded":"2023-10-06T11:22:01.621178+0000","last_fullsized":"2023-10-06T11:22:01.621178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610375+0000","last_change":"2023-10-06T11:21:57.575519+0000","last_active":"2023-10-06T11:22:01.610375+0000","last_peered":"2023-10-06T11:22:01.610375+0000","last_clean":"2023-10-06T11:22:01.610375+0000","last_became_active":"2023-10-06T11:21:57.575122+0000","last_became_peered":"2023-10-06T11:21:57.575122+0000","last_unstale":"2023-10-06T11:22:01.610375+0000","last_undegraded":"2023-10-06T11:22:01.610375+0000","last_fullsized":"2023-10-06T11:22:01.610375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:01.582258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607465+0000","last_change":"2023-10-06T11:21:57.563806+0000","last_active":"2023-10-06T11:22:01.607465+0000","last_peered":"2023-10-06T11:22:01.607465+0000","last_clean":"2023-10-06T11:22:01.607465+0000","last_became_active":"2023-10-06T11:21:57.561796+0000","last_became_peered":"2023-10-06T11:21:57.561796+0000","last_unstale":"2023-10-06T11:22:01.607465+0000","last_undegraded":"2023-10-06T11:22:01.607465+0000","last_fullsized":"2023-10-06T11:22:01.607465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610200+0000","last_change":"2023-10-06T11:21:57.560661+0000","last_active":"2023-10-06T11:22:01.610200+0000","last_peered":"2023-10-06T11:22:01.610200+0000","last_clean":"2023-10-06T11:22:01.610200+0000","last_became_active":"2023-10-06T11:21:57.559848+0000","last_became_peered":"2023-10-06T11:21:57.559848+0000","last_unstale":"2023-10-06T11:22:01.610200+0000","last_undegraded":"2023-10-06T11:22:01.610200+0000","last_fullsized":"2023-10-06T11:22:01.610200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621548+0000","last_change":"2023-10-06T11:21:57.566423+0000","last_active":"2023-10-06T11:22:01.621548+0000","last_peered":"2023-10-06T11:22:01.621548+0000","last_clean":"2023-10-06T11:22:01.621548+0000","last_became_active":"2023-10-06T11:21:57.566064+0000","last_became_peered":"2023-10-06T11:21:57.566064+0000","last_unstale":"2023-10-06T11:22:01.621548+0000","last_undegraded":"2023-10-06T11:22:01.621548+0000","last_fullsized":"2023-10-06T11:22:01.621548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627537+0000","last_change":"2023-10-06T11:21:57.575695+0000","last_active":"2023-10-06T11:22:01.627537+0000","last_peered":"2023-10-06T11:22:01.627537+0000","last_clean":"2023-10-06T11:22:01.627537+0000","last_became_active":"2023-10-06T11:21:57.575418+0000","last_became_peered":"2023-10-06T11:21:57.575418+0000","last_unstale":"2023-10-06T11:22:01.627537+0000","last_undegraded":"2023-10-06T11:22:01.627537+0000","last_fullsized":"2023-10-06T11:22:01.627537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619861+0000","last_change":"2023-10-06T11:21:57.562165+0000","last_active":"2023-10-06T11:22:01.619861+0000","last_peered":"2023-10-06T11:22:01.619861+0000","last_clean":"2023-10-06T11:22:01.619861+0000","last_became_active":"2023-10-06T11:21:57.561878+0000","last_became_peered":"2023-10-06T11:21:57.561878+0000","last_unstale":"2023-10-06T11:22:01.619861+0000","last_undegraded":"2023-10-06T11:22:01.619861+0000","last_fullsized":"2023-10-06T11:22:01.619861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619664+0000","last_change":"2023-10-06T11:21:57.575950+0000","last_active":"2023-10-06T11:22:01.619664+0000","last_peered":"2023-10-06T11:22:01.619664+0000","last_clean":"2023-10-06T11:22:01.619664+0000","last_became_active":"2023-10-06T11:21:57.575280+0000","last_became_peered":"2023-10-06T11:21:57.575280+0000","last_unstale":"2023-10-06T11:22:01.619664+0000","last_undegraded":"2023-10-06T11:22:01.619664+0000","last_fullsized":"2023-10-06T11:22:01.619664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609933+0000","last_change":"2023-10-06T11:21:57.566684+0000","last_active":"2023-10-06T11:22:01.609933+0000","last_peered":"2023-10-06T11:22:01.609933+0000","last_clean":"2023-10-06T11:22:01.609933+0000","last_became_active":"2023-10-06T11:21:57.566358+0000","last_became_peered":"2023-10-06T11:21:57.566358+0000","last_unstale":"2023-10-06T11:22:01.609933+0000","last_undegraded":"2023-10-06T11:22:01.609933+0000","last_fullsized":"2023-10-06T11:22:01.609933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:50.324902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626719+0000","last_change":"2023-10-06T11:21:57.574099+0000","last_active":"2023-10-06T11:22:01.626719+0000","last_peered":"2023-10-06T11:22:01.626719+0000","last_clean":"2023-10-06T11:22:01.626719+0000","last_became_active":"2023-10-06T11:21:57.573592+0000","last_became_peered":"2023-10-06T11:21:57.573592+0000","last_unstale":"2023-10-06T11:22:01.626719+0000","last_undegraded":"2023-10-06T11:22:01.626719+0000","last_fullsized":"2023-10-06T11:22:01.626719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619959+0000","last_change":"2023-10-06T11:21:57.566310+0000","last_active":"2023-10-06T11:22:01.619959+0000","last_peered":"2023-10-06T11:22:01.619959+0000","last_clean":"2023-10-06T11:22:01.619959+0000","last_became_active":"2023-10-06T11:21:57.566014+0000","last_became_peered":"2023-10-06T11:21:57.566014+0000","last_unstale":"2023-10-06T11:22:01.619959+0000","last_undegraded":"2023-10-06T11:22:01.619959+0000","last_fullsized":"2023-10-06T11:22:01.619959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.268101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609857+0000","last_change":"2023-10-06T11:21:57.567982+0000","last_active":"2023-10-06T11:22:01.609857+0000","last_peered":"2023-10-06T11:22:01.609857+0000","last_clean":"2023-10-06T11:22:01.609857+0000","last_became_active":"2023-10-06T11:21:57.567667+0000","last_became_peered":"2023-10-06T11:21:57.567667+0000","last_unstale":"2023-10-06T11:22:01.609857+0000","last_undegraded":"2023-10-06T11:22:01.609857+0000","last_fullsized":"2023-10-06T11:22:01.609857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:14.276554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621434+0000","last_change":"2023-10-06T11:21:57.562657+0000","last_active":"2023-10-06T11:22:01.621434+0000","last_peered":"2023-10-06T11:22:01.621434+0000","last_clean":"2023-10-06T11:22:01.621434+0000","last_became_active":"2023-10-06T11:21:57.562338+0000","last_became_peered":"2023-10-06T11:21:57.562338+0000","last_unstale":"2023-10-06T11:22:01.621434+0000","last_undegraded":"2023-10-06T11:22:01.621434+0000","last_fullsized":"2023-10-06T11:22:01.621434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607967+0000","last_change":"2023-10-06T11:21:57.569451+0000","last_active":"2023-10-06T11:22:01.607967+0000","last_peered":"2023-10-06T11:22:01.607967+0000","last_clean":"2023-10-06T11:22:01.607967+0000","last_became_active":"2023-10-06T11:21:57.569124+0000","last_became_peered":"2023-10-06T11:21:57.569124+0000","last_unstale":"2023-10-06T11:22:01.607967+0000","last_undegraded":"2023-10-06T11:22:01.607967+0000","last_fullsized":"2023-10-06T11:22:01.607967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.833392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":703756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301652,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070188,"statfs":{"total":96636764160,"available":96327872512,"internally_reserved":0,"allocated":274432,"data_stored":110082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":286720,"data_stored":113228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":109548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302224,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069616,"statfs":{"total":96636764160,"available":96327286784,"internally_reserved":0,"allocated":860160,"data_stored":699826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":110012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:22:17.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:22:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 Processor -- start 2023-10-06T11:22:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 -- start start 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 0x7f7a8c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c0ca240 con 0x7f7a8c06c320 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c15ad00 con 0x7f7a8c156af0 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a947b6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0ae560 con 0x7f7a8c0adf80 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a91d51700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a8c06c320 0x7f7a8c0ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54770/0 (socket says 172.21.15.7:54770) 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.359+0000 7f7a91d51700 1 -- 172.21.15.7:0/3521554416 learned_addr learned my addr 172.21.15.7:0/3521554416 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:17.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a92d53700 1 --2- 172.21.15.7:0/3521554416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 0x7f7a8c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568669016 0 0) 0x7f7a8c0ca240 con 0x7f7a8c06c320 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a78003570 con 0x7f7a8c06c320 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297854125 0 0) 0x7f7a8c15ad00 con 0x7f7a8c156af0 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8c0ca240 con 0x7f7a8c156af0 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a92d53700 1 -- 172.21.15.7:0/3521554416 >> v1:172.21.15.50:6789/0 conn(0x7f7a8c156af0 legacy=0x7f7a8c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a92d53700 1 -- 172.21.15.7:0/3521554416 >> v1:172.21.15.7:6789/0 conn(0x7f7a8c06c320 legacy=0x7f7a8c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a92d53700 1 -- 172.21.15.7:0/3521554416 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c15afe0 con 0x7f7a8c0adf80 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a92d53700 1 --2- 172.21.15.7:0/3521554416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 0x7f7a8c156ec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009ec0 tx=0x7f7a84009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496b8436bdbd6d29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400e070 con 0x7f7a8c0adf80 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8400bb40 con 0x7f7a8c0adf80 2023-10-06T11:22:17.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a91550700 1 -- 172.21.15.7:0/3521554416 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84012430 con 0x7f7a8c0adf80 2023-10-06T11:22:17.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 -- 172.21.15.7:0/3521554416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 msgr2=0x7f7a8c156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 --2- 172.21.15.7:0/3521554416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 0x7f7a8c156ec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009ec0 tx=0x7f7a84009780 comp rx=0 tx=0).stop 2023-10-06T11:22:17.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 -- 172.21.15.7:0/3521554416 shutdown_connections 2023-10-06T11:22:17.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 --2- 172.21.15.7:0/3521554416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c0adf80 0x7f7a8c156ec0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 -- 172.21.15.7:0/3521554416 >> 172.21.15.7:0/3521554416 conn(0x7f7a8c0cdc90 msgr2=0x7f7a8c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 -- 172.21.15.7:0/3521554416 shutdown_connections 2023-10-06T11:22:17.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.363+0000 7f7a947b6700 1 -- 172.21.15.7:0/3521554416 wait complete. 2023-10-06T11:22:17.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 Processor -- start 2023-10-06T11:22:17.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 -- start start 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 0x7f7a8c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 0x7f7a8c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c156af0 0x7f7a8c0c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c16a200 con 0x7f7a8c0adf80 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c16a4e0 con 0x7f7a8c156af0 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a947b6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c16a7c0 con 0x7f7a8c06c320 2023-10-06T11:22:17.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a91d51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 0x7f7a8c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a91d51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 0x7f7a8c0c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41528/0 (socket says 172.21.15.7:41528) 2023-10-06T11:22:17.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a91d51700 1 -- 172.21.15.7:0/1326739958 learned_addr learned my addr 172.21.15.7:0/1326739958 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:17.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a92552700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 0x7f7a8c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.367+0000 7f7a92d53700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c156af0 0x7f7a8c0c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 -- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c156af0 msgr2=0x7f7a8c0c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c156af0 0x7f7a8c0c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 -- 172.21.15.7:0/1326739958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 msgr2=0x7f7a8c0c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 0x7f7a8c0c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 -- 172.21.15.7:0/1326739958 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009a20 con 0x7f7a8c06c320 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a92552700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 0x7f7a8c1557c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00ce10 con 0x7f7a8c06c320 2023-10-06T11:22:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c011590 con 0x7f7a8c06c320 2023-10-06T11:22:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c16aaa0 con 0x7f7a8c06c320 2023-10-06T11:22:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c011950 con 0x7f7a8c06c320 2023-10-06T11:22:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c0c7110 con 0x7f7a8c06c320 2023-10-06T11:22:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c16afa0 con 0x7f7a8c06c320 2023-10-06T11:22:17.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.371+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7a7c011d40 con 0x7f7a8c06c320 2023-10-06T11:22:17.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.375+0000 7f7a827fc700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 0x7f7a70062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.375+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7a7c07c710 con 0x7f7a8c06c320 2023-10-06T11:22:17.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.375+0000 7f7a91d51700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 0x7f7a70062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.375+0000 7f7a91d51700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 0x7f7a70062a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0adb80 tx=0x7f7a88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.383+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7a7c04c8b0 con 0x7f7a8c06c320 2023-10-06T11:22:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.547+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a8c04be00 con 0x7f7a70060550 2023-10-06T11:22:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.567+0000 7f7a827fc700 1 -- 172.21.15.7:0/1326739958 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367453 (secure 0 0 0) 0x7f7a8c04be00 con 0x7f7a70060550 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 msgr2=0x7f7a70062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 0x7f7a70062a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0adb80 tx=0x7f7a88009040 comp rx=0 tx=0).stop 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 msgr2=0x7f7a8c1557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 0x7f7a8c1557c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 shutdown_connections 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7a70060550 0x7f7a70062a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c156af0 0x7f7a8c0c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8c0adf80 0x7f7a8c0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 --2- 172.21.15.7:0/1326739958 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7a8c06c320 0x7f7a8c1557c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 >> 172.21.15.7:0/1326739958 conn(0x7f7a8c0cdc90 msgr2=0x7f7a8c1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 shutdown_connections 2023-10-06T11:22:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.575+0000 7f7a947b6700 1 -- 172.21.15.7:0/1326739958 wait complete. 2023-10-06T11:22:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:22:17.593 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-06T11:22:15.816921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414376,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752560344,"statfs":{"total":773094113280,"available":770621792256,"internally_reserved":0,"allocated":3383296,"data_stored":2065368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005022"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642766+0000","last_change":"2023-10-06T11:22:01.634221+0000","last_active":"2023-10-06T11:22:01.642766+0000","last_peered":"2023-10-06T11:22:01.642766+0000","last_clean":"2023-10-06T11:22:01.642766+0000","last_became_active":"2023-10-06T11:22:01.633727+0000","last_became_peered":"2023-10-06T11:22:01.633727+0000","last_unstale":"2023-10-06T11:22:01.642766+0000","last_undegraded":"2023-10-06T11:22:01.642766+0000","last_fullsized":"2023-10-06T11:22:01.642766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627705+0000","last_change":"2023-10-06T11:21:56.568541+0000","last_active":"2023-10-06T11:22:01.627705+0000","last_peered":"2023-10-06T11:22:01.627705+0000","last_clean":"2023-10-06T11:22:01.627705+0000","last_became_active":"2023-10-06T11:21:56.567104+0000","last_became_peered":"2023-10-06T11:21:56.567104+0000","last_unstale":"2023-10-06T11:22:01.627705+0000","last_undegraded":"2023-10-06T11:22:01.627705+0000","last_fullsized":"2023-10-06T11:22:01.627705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612705+0000","last_change":"2023-10-06T11:21:57.566064+0000","last_active":"2023-10-06T11:22:01.612705+0000","last_peered":"2023-10-06T11:22:01.612705+0000","last_clean":"2023-10-06T11:22:01.612705+0000","last_became_active":"2023-10-06T11:21:57.565589+0000","last_became_peered":"2023-10-06T11:21:57.565589+0000","last_unstale":"2023-10-06T11:22:01.612705+0000","last_undegraded":"2023-10-06T11:22:01.612705+0000","last_fullsized":"2023-10-06T11:22:01.612705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645194+0000","last_change":"2023-10-06T11:22:01.637184+0000","last_active":"2023-10-06T11:22:01.645194+0000","last_peered":"2023-10-06T11:22:01.645194+0000","last_clean":"2023-10-06T11:22:01.645194+0000","last_became_active":"2023-10-06T11:22:01.636773+0000","last_became_peered":"2023-10-06T11:22:01.636773+0000","last_unstale":"2023-10-06T11:22:01.645194+0000","last_undegraded":"2023-10-06T11:22:01.645194+0000","last_fullsized":"2023-10-06T11:22:01.645194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:41.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617318+0000","last_change":"2023-10-06T11:21:56.565259+0000","last_active":"2023-10-06T11:22:01.617318+0000","last_peered":"2023-10-06T11:22:01.617318+0000","last_clean":"2023-10-06T11:22:01.617318+0000","last_became_active":"2023-10-06T11:21:56.564927+0000","last_became_peered":"2023-10-06T11:21:56.564927+0000","last_unstale":"2023-10-06T11:22:01.617318+0000","last_undegraded":"2023-10-06T11:22:01.617318+0000","last_fullsized":"2023-10-06T11:22:01.617318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617577+0000","last_change":"2023-10-06T11:21:57.562278+0000","last_active":"2023-10-06T11:22:01.617577+0000","last_peered":"2023-10-06T11:22:01.617577+0000","last_clean":"2023-10-06T11:22:01.617577+0000","last_became_active":"2023-10-06T11:21:57.561972+0000","last_became_peered":"2023-10-06T11:21:57.561972+0000","last_unstale":"2023-10-06T11:22:01.617577+0000","last_undegraded":"2023-10-06T11:22:01.617577+0000","last_fullsized":"2023-10-06T11:22:01.617577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641228+0000","last_change":"2023-10-06T11:22:01.637805+0000","last_active":"2023-10-06T11:22:01.641228+0000","last_peered":"2023-10-06T11:22:01.641228+0000","last_clean":"2023-10-06T11:22:01.641228+0000","last_became_active":"2023-10-06T11:22:01.637404+0000","last_became_peered":"2023-10-06T11:22:01.637404+0000","last_unstale":"2023-10-06T11:22:01.641228+0000","last_undegraded":"2023-10-06T11:22:01.641228+0000","last_fullsized":"2023-10-06T11:22:01.641228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621633+0000","last_change":"2023-10-06T11:21:56.578707+0000","last_active":"2023-10-06T11:22:01.621633+0000","last_peered":"2023-10-06T11:22:01.621633+0000","last_clean":"2023-10-06T11:22:01.621633+0000","last_became_active":"2023-10-06T11:21:56.578411+0000","last_became_peered":"2023-10-06T11:21:56.578411+0000","last_unstale":"2023-10-06T11:22:01.621633+0000","last_undegraded":"2023-10-06T11:22:01.621633+0000","last_fullsized":"2023-10-06T11:22:01.621633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610166+0000","last_change":"2023-10-06T11:21:57.570591+0000","last_active":"2023-10-06T11:22:01.610166+0000","last_peered":"2023-10-06T11:22:01.610166+0000","last_clean":"2023-10-06T11:22:01.610166+0000","last_became_active":"2023-10-06T11:21:57.570166+0000","last_became_peered":"2023-10-06T11:21:57.570166+0000","last_unstale":"2023-10-06T11:22:01.610166+0000","last_undegraded":"2023-10-06T11:22:01.610166+0000","last_fullsized":"2023-10-06T11:22:01.610166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644696+0000","last_change":"2023-10-06T11:22:01.638015+0000","last_active":"2023-10-06T11:22:01.644696+0000","last_peered":"2023-10-06T11:22:01.644696+0000","last_clean":"2023-10-06T11:22:01.644696+0000","last_became_active":"2023-10-06T11:22:01.637477+0000","last_became_peered":"2023-10-06T11:22:01.637477+0000","last_unstale":"2023-10-06T11:22:01.644696+0000","last_undegraded":"2023-10-06T11:22:01.644696+0000","last_fullsized":"2023-10-06T11:22:01.644696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:56.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621240+0000","last_change":"2023-10-06T11:21:56.571616+0000","last_active":"2023-10-06T11:22:01.621240+0000","last_peered":"2023-10-06T11:22:01.621240+0000","last_clean":"2023-10-06T11:22:01.621240+0000","last_became_active":"2023-10-06T11:21:56.571289+0000","last_became_peered":"2023-10-06T11:21:56.571289+0000","last_unstale":"2023-10-06T11:22:01.621240+0000","last_undegraded":"2023-10-06T11:22:01.621240+0000","last_fullsized":"2023-10-06T11:22:01.621240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611658+0000","last_change":"2023-10-06T11:21:57.575085+0000","last_active":"2023-10-06T11:22:01.611658+0000","last_peered":"2023-10-06T11:22:01.611658+0000","last_clean":"2023-10-06T11:22:01.611658+0000","last_became_active":"2023-10-06T11:21:57.574716+0000","last_became_peered":"2023-10-06T11:21:57.574716+0000","last_unstale":"2023-10-06T11:22:01.611658+0000","last_undegraded":"2023-10-06T11:22:01.611658+0000","last_fullsized":"2023-10-06T11:22:01.611658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643969+0000","last_change":"2023-10-06T11:22:01.638673+0000","last_active":"2023-10-06T11:22:01.643969+0000","last_peered":"2023-10-06T11:22:01.643969+0000","last_clean":"2023-10-06T11:22:01.643969+0000","last_became_active":"2023-10-06T11:22:01.635630+0000","last_became_peered":"2023-10-06T11:22:01.635630+0000","last_unstale":"2023-10-06T11:22:01.643969+0000","last_undegraded":"2023-10-06T11:22:01.643969+0000","last_fullsized":"2023-10-06T11:22:01.643969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626800+0000","last_change":"2023-10-06T11:21:56.572349+0000","last_active":"2023-10-06T11:22:01.626800+0000","last_peered":"2023-10-06T11:22:01.626800+0000","last_clean":"2023-10-06T11:22:01.626800+0000","last_became_active":"2023-10-06T11:21:56.571848+0000","last_became_peered":"2023-10-06T11:21:56.571848+0000","last_unstale":"2023-10-06T11:22:01.626800+0000","last_undegraded":"2023-10-06T11:22:01.626800+0000","last_fullsized":"2023-10-06T11:22:01.626800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607694+0000","last_change":"2023-10-06T11:21:57.580595+0000","last_active":"2023-10-06T11:22:01.607694+0000","last_peered":"2023-10-06T11:22:01.607694+0000","last_clean":"2023-10-06T11:22:01.607694+0000","last_became_active":"2023-10-06T11:21:57.580286+0000","last_became_peered":"2023-10-06T11:21:57.580286+0000","last_unstale":"2023-10-06T11:22:01.607694+0000","last_undegraded":"2023-10-06T11:22:01.607694+0000","last_fullsized":"2023-10-06T11:22:01.607694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:04.725222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642661+0000","last_change":"2023-10-06T11:22:01.632283+0000","last_active":"2023-10-06T11:22:01.642661+0000","last_peered":"2023-10-06T11:22:01.642661+0000","last_clean":"2023-10-06T11:22:01.642661+0000","last_became_active":"2023-10-06T11:22:01.628222+0000","last_became_peered":"2023-10-06T11:22:01.628222+0000","last_unstale":"2023-10-06T11:22:01.642661+0000","last_undegraded":"2023-10-06T11:22:01.642661+0000","last_fullsized":"2023-10-06T11:22:01.642661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:32.168166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610453+0000","last_change":"2023-10-06T11:21:56.571031+0000","last_active":"2023-10-06T11:22:01.610453+0000","last_peered":"2023-10-06T11:22:01.610453+0000","last_clean":"2023-10-06T11:22:01.610453+0000","last_became_active":"2023-10-06T11:21:56.570724+0000","last_became_peered":"2023-10-06T11:21:56.570724+0000","last_unstale":"2023-10-06T11:22:01.610453+0000","last_undegraded":"2023-10-06T11:22:01.610453+0000","last_fullsized":"2023-10-06T11:22:01.610453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616088+0000","last_change":"2023-10-06T11:21:57.562757+0000","last_active":"2023-10-06T11:22:01.616088+0000","last_peered":"2023-10-06T11:22:01.616088+0000","last_clean":"2023-10-06T11:22:01.616088+0000","last_became_active":"2023-10-06T11:21:57.562240+0000","last_became_peered":"2023-10-06T11:21:57.562240+0000","last_unstale":"2023-10-06T11:22:01.616088+0000","last_undegraded":"2023-10-06T11:22:01.616088+0000","last_fullsized":"2023-10-06T11:22:01.616088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.636503+0000","last_change":"2023-10-06T11:22:01.635655+0000","last_active":"2023-10-06T11:22:01.636503+0000","last_peered":"2023-10-06T11:22:01.636503+0000","last_clean":"2023-10-06T11:22:01.636503+0000","last_became_active":"2023-10-06T11:22:01.635167+0000","last_became_peered":"2023-10-06T11:22:01.635167+0000","last_unstale":"2023-10-06T11:22:01.636503+0000","last_undegraded":"2023-10-06T11:22:01.636503+0000","last_fullsized":"2023-10-06T11:22:01.636503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:16.080135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617232+0000","last_change":"2023-10-06T11:21:56.569415+0000","last_active":"2023-10-06T11:22:01.617232+0000","last_peered":"2023-10-06T11:22:01.617232+0000","last_clean":"2023-10-06T11:22:01.617232+0000","last_became_active":"2023-10-06T11:21:56.569132+0000","last_became_peered":"2023-10-06T11:21:56.569132+0000","last_unstale":"2023-10-06T11:22:01.617232+0000","last_undegraded":"2023-10-06T11:22:01.617232+0000","last_fullsized":"2023-10-06T11:22:01.617232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627493+0000","last_change":"2023-10-06T11:21:57.566844+0000","last_active":"2023-10-06T11:22:01.627493+0000","last_peered":"2023-10-06T11:22:01.627493+0000","last_clean":"2023-10-06T11:22:01.627493+0000","last_became_active":"2023-10-06T11:21:57.566564+0000","last_became_peered":"2023-10-06T11:21:57.566564+0000","last_unstale":"2023-10-06T11:22:01.627493+0000","last_undegraded":"2023-10-06T11:22:01.627493+0000","last_fullsized":"2023-10-06T11:22:01.627493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640584+0000","last_change":"2023-10-06T11:22:01.639744+0000","last_active":"2023-10-06T11:22:01.640584+0000","last_peered":"2023-10-06T11:22:01.640584+0000","last_clean":"2023-10-06T11:22:01.640584+0000","last_became_active":"2023-10-06T11:22:01.639253+0000","last_became_peered":"2023-10-06T11:22:01.639253+0000","last_unstale":"2023-10-06T11:22:01.640584+0000","last_undegraded":"2023-10-06T11:22:01.640584+0000","last_fullsized":"2023-10-06T11:22:01.640584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615560+0000","last_change":"2023-10-06T11:21:56.574525+0000","last_active":"2023-10-06T11:22:01.615560+0000","last_peered":"2023-10-06T11:22:01.615560+0000","last_clean":"2023-10-06T11:22:01.615560+0000","last_became_active":"2023-10-06T11:21:56.574182+0000","last_became_peered":"2023-10-06T11:21:56.574182+0000","last_unstale":"2023-10-06T11:22:01.615560+0000","last_undegraded":"2023-10-06T11:22:01.615560+0000","last_fullsized":"2023-10-06T11:22:01.615560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620100+0000","last_change":"2023-10-06T11:21:57.564759+0000","last_active":"2023-10-06T11:22:01.620100+0000","last_peered":"2023-10-06T11:22:01.620100+0000","last_clean":"2023-10-06T11:22:01.620100+0000","last_became_active":"2023-10-06T11:21:57.564385+0000","last_became_peered":"2023-10-06T11:21:57.564385+0000","last_unstale":"2023-10-06T11:22:01.620100+0000","last_undegraded":"2023-10-06T11:22:01.620100+0000","last_fullsized":"2023-10-06T11:22:01.620100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615364+0000","last_change":"2023-10-06T11:21:57.567825+0000","last_active":"2023-10-06T11:22:01.615364+0000","last_peered":"2023-10-06T11:22:01.615364+0000","last_clean":"2023-10-06T11:22:01.615364+0000","last_became_active":"2023-10-06T11:21:57.562342+0000","last_became_peered":"2023-10-06T11:21:57.562342+0000","last_unstale":"2023-10-06T11:22:01.615364+0000","last_undegraded":"2023-10-06T11:22:01.615364+0000","last_fullsized":"2023-10-06T11:22:01.615364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627753+0000","last_change":"2023-10-06T11:21:56.573919+0000","last_active":"2023-10-06T11:22:01.627753+0000","last_peered":"2023-10-06T11:22:01.627753+0000","last_clean":"2023-10-06T11:22:01.627753+0000","last_became_active":"2023-10-06T11:21:56.573605+0000","last_became_peered":"2023-10-06T11:21:56.573605+0000","last_unstale":"2023-10-06T11:22:01.627753+0000","last_undegraded":"2023-10-06T11:22:01.627753+0000","last_fullsized":"2023-10-06T11:22:01.627753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639931+0000","last_change":"2023-10-06T11:22:01.637440+0000","last_active":"2023-10-06T11:22:01.639931+0000","last_peered":"2023-10-06T11:22:01.639931+0000","last_clean":"2023-10-06T11:22:01.639931+0000","last_became_active":"2023-10-06T11:22:01.637003+0000","last_became_peered":"2023-10-06T11:22:01.637003+0000","last_unstale":"2023-10-06T11:22:01.639931+0000","last_undegraded":"2023-10-06T11:22:01.639931+0000","last_fullsized":"2023-10-06T11:22:01.639931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.578282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617665+0000","last_change":"2023-10-06T11:21:57.572470+0000","last_active":"2023-10-06T11:22:01.617665+0000","last_peered":"2023-10-06T11:22:01.617665+0000","last_clean":"2023-10-06T11:22:01.617665+0000","last_became_active":"2023-10-06T11:21:57.568699+0000","last_became_peered":"2023-10-06T11:21:57.568699+0000","last_unstale":"2023-10-06T11:22:01.617665+0000","last_undegraded":"2023-10-06T11:22:01.617665+0000","last_fullsized":"2023-10-06T11:22:01.617665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621640+0000","last_change":"2023-10-06T11:21:56.576266+0000","last_active":"2023-10-06T11:22:01.621640+0000","last_peered":"2023-10-06T11:22:01.621640+0000","last_clean":"2023-10-06T11:22:01.621640+0000","last_became_active":"2023-10-06T11:21:56.575879+0000","last_became_peered":"2023-10-06T11:21:56.575879+0000","last_unstale":"2023-10-06T11:22:01.621640+0000","last_undegraded":"2023-10-06T11:22:01.621640+0000","last_fullsized":"2023-10-06T11:22:01.621640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645265+0000","last_change":"2023-10-06T11:22:01.642796+0000","last_active":"2023-10-06T11:22:01.645265+0000","last_peered":"2023-10-06T11:22:01.645265+0000","last_clean":"2023-10-06T11:22:01.645265+0000","last_became_active":"2023-10-06T11:22:01.642102+0000","last_became_peered":"2023-10-06T11:22:01.642102+0000","last_unstale":"2023-10-06T11:22:01.645265+0000","last_undegraded":"2023-10-06T11:22:01.645265+0000","last_fullsized":"2023-10-06T11:22:01.645265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620311+0000","last_change":"2023-10-06T11:21:56.577336+0000","last_active":"2023-10-06T11:22:01.620311+0000","last_peered":"2023-10-06T11:22:01.620311+0000","last_clean":"2023-10-06T11:22:01.620311+0000","last_became_active":"2023-10-06T11:21:56.574934+0000","last_became_peered":"2023-10-06T11:21:56.574934+0000","last_unstale":"2023-10-06T11:22:01.620311+0000","last_undegraded":"2023-10-06T11:22:01.620311+0000","last_fullsized":"2023-10-06T11:22:01.620311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:58.978806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615123+0000","last_change":"2023-10-06T11:21:57.565736+0000","last_active":"2023-10-06T11:22:01.615123+0000","last_peered":"2023-10-06T11:22:01.615123+0000","last_clean":"2023-10-06T11:22:01.615123+0000","last_became_active":"2023-10-06T11:21:57.565078+0000","last_became_peered":"2023-10-06T11:21:57.565078+0000","last_unstale":"2023-10-06T11:22:01.615123+0000","last_undegraded":"2023-10-06T11:22:01.615123+0000","last_fullsized":"2023-10-06T11:22:01.615123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637186+0000","last_change":"2023-10-06T11:22:01.635715+0000","last_active":"2023-10-06T11:22:01.637186+0000","last_peered":"2023-10-06T11:22:01.637186+0000","last_clean":"2023-10-06T11:22:01.637186+0000","last_became_active":"2023-10-06T11:22:01.635213+0000","last_became_peered":"2023-10-06T11:22:01.635213+0000","last_unstale":"2023-10-06T11:22:01.637186+0000","last_undegraded":"2023-10-06T11:22:01.637186+0000","last_fullsized":"2023-10-06T11:22:01.637186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:00.176987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-06T11:22:07.460343+0000","last_change":"2023-10-06T11:21:56.573406+0000","last_active":"2023-10-06T11:22:07.460343+0000","last_peered":"2023-10-06T11:22:07.460343+0000","last_clean":"2023-10-06T11:22:07.460343+0000","last_became_active":"2023-10-06T11:21:56.572858+0000","last_became_peered":"2023-10-06T11:21:56.572858+0000","last_unstale":"2023-10-06T11:22:07.460343+0000","last_undegraded":"2023-10-06T11:22:07.460343+0000","last_fullsized":"2023-10-06T11:22:07.460343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611768+0000","last_change":"2023-10-06T11:21:57.566510+0000","last_active":"2023-10-06T11:22:01.611768+0000","last_peered":"2023-10-06T11:22:01.611768+0000","last_clean":"2023-10-06T11:22:01.611768+0000","last_became_active":"2023-10-06T11:21:57.566168+0000","last_became_peered":"2023-10-06T11:21:57.566168+0000","last_unstale":"2023-10-06T11:22:01.611768+0000","last_undegraded":"2023-10-06T11:22:01.611768+0000","last_fullsized":"2023-10-06T11:22:01.611768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:03.939980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644970+0000","last_change":"2023-10-06T11:22:01.640707+0000","last_active":"2023-10-06T11:22:01.644970+0000","last_peered":"2023-10-06T11:22:01.644970+0000","last_clean":"2023-10-06T11:22:01.644970+0000","last_became_active":"2023-10-06T11:22:01.640294+0000","last_became_peered":"2023-10-06T11:22:01.640294+0000","last_unstale":"2023-10-06T11:22:01.644970+0000","last_undegraded":"2023-10-06T11:22:01.644970+0000","last_fullsized":"2023-10-06T11:22:01.644970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:07.995705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615237+0000","last_change":"2023-10-06T11:21:57.565357+0000","last_active":"2023-10-06T11:22:01.615237+0000","last_peered":"2023-10-06T11:22:01.615237+0000","last_clean":"2023-10-06T11:22:01.615237+0000","last_became_active":"2023-10-06T11:21:57.560887+0000","last_became_peered":"2023-10-06T11:21:57.560887+0000","last_unstale":"2023-10-06T11:22:01.615237+0000","last_undegraded":"2023-10-06T11:22:01.615237+0000","last_fullsized":"2023-10-06T11:22:01.615237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620136+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:22:01.620136+0000","last_peered":"2023-10-06T11:22:01.620136+0000","last_clean":"2023-10-06T11:22:01.620136+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:22:01.620136+0000","last_undegraded":"2023-10-06T11:22:01.620136+0000","last_fullsized":"2023-10-06T11:22:01.620136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621725+0000","last_change":"2023-10-06T11:21:56.569726+0000","last_active":"2023-10-06T11:22:01.621725+0000","last_peered":"2023-10-06T11:22:01.621725+0000","last_clean":"2023-10-06T11:22:01.621725+0000","last_became_active":"2023-10-06T11:21:56.569394+0000","last_became_peered":"2023-10-06T11:21:56.569394+0000","last_unstale":"2023-10-06T11:22:01.621725+0000","last_undegraded":"2023-10-06T11:22:01.621725+0000","last_fullsized":"2023-10-06T11:22:01.621725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640548+0000","last_change":"2023-10-06T11:22:01.637613+0000","last_active":"2023-10-06T11:22:01.640548+0000","last_peered":"2023-10-06T11:22:01.640548+0000","last_clean":"2023-10-06T11:22:01.640548+0000","last_became_active":"2023-10-06T11:22:01.636818+0000","last_became_peered":"2023-10-06T11:22:01.636818+0000","last_unstale":"2023-10-06T11:22:01.640548+0000","last_undegraded":"2023-10-06T11:22:01.640548+0000","last_fullsized":"2023-10-06T11:22:01.640548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626583+0000","last_change":"2023-10-06T11:21:57.556132+0000","last_active":"2023-10-06T11:22:01.626583+0000","last_peered":"2023-10-06T11:22:01.626583+0000","last_clean":"2023-10-06T11:22:01.626583+0000","last_became_active":"2023-10-06T11:21:57.555766+0000","last_became_peered":"2023-10-06T11:21:57.555766+0000","last_unstale":"2023-10-06T11:22:01.626583+0000","last_undegraded":"2023-10-06T11:22:01.626583+0000","last_fullsized":"2023-10-06T11:22:01.626583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620468+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:22:01.620468+0000","last_peered":"2023-10-06T11:22:01.620468+0000","last_clean":"2023-10-06T11:22:01.620468+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:22:01.620468+0000","last_undegraded":"2023-10-06T11:22:01.620468+0000","last_fullsized":"2023-10-06T11:22:01.620468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619168+0000","last_change":"2023-10-06T11:21:56.570891+0000","last_active":"2023-10-06T11:22:01.619168+0000","last_peered":"2023-10-06T11:22:01.619168+0000","last_clean":"2023-10-06T11:22:01.619168+0000","last_became_active":"2023-10-06T11:21:56.570569+0000","last_became_peered":"2023-10-06T11:21:56.570569+0000","last_unstale":"2023-10-06T11:22:01.619168+0000","last_undegraded":"2023-10-06T11:22:01.619168+0000","last_fullsized":"2023-10-06T11:22:01.619168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634770+0000","last_change":"2023-10-06T11:22:01.630420+0000","last_active":"2023-10-06T11:22:01.634770+0000","last_peered":"2023-10-06T11:22:01.634770+0000","last_clean":"2023-10-06T11:22:01.634770+0000","last_became_active":"2023-10-06T11:22:01.629842+0000","last_became_peered":"2023-10-06T11:22:01.629842+0000","last_unstale":"2023-10-06T11:22:01.634770+0000","last_undegraded":"2023-10-06T11:22:01.634770+0000","last_fullsized":"2023-10-06T11:22:01.634770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:49.797072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615458+0000","last_change":"2023-10-06T11:21:57.568394+0000","last_active":"2023-10-06T11:22:01.615458+0000","last_peered":"2023-10-06T11:22:01.615458+0000","last_clean":"2023-10-06T11:22:01.615458+0000","last_became_active":"2023-10-06T11:21:57.568120+0000","last_became_peered":"2023-10-06T11:21:57.568120+0000","last_unstale":"2023-10-06T11:22:01.615458+0000","last_undegraded":"2023-10-06T11:22:01.615458+0000","last_fullsized":"2023-10-06T11:22:01.615458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610919+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:22:01.610919+0000","last_peered":"2023-10-06T11:22:01.610919+0000","last_clean":"2023-10-06T11:22:01.610919+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:22:01.610919+0000","last_undegraded":"2023-10-06T11:22:01.610919+0000","last_fullsized":"2023-10-06T11:22:01.610919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616279+0000","last_change":"2023-10-06T11:21:56.574787+0000","last_active":"2023-10-06T11:22:01.616279+0000","last_peered":"2023-10-06T11:22:01.616279+0000","last_clean":"2023-10-06T11:22:01.616279+0000","last_became_active":"2023-10-06T11:21:56.574471+0000","last_became_peered":"2023-10-06T11:21:56.574471+0000","last_unstale":"2023-10-06T11:22:01.616279+0000","last_undegraded":"2023-10-06T11:22:01.616279+0000","last_fullsized":"2023-10-06T11:22:01.616279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642714+0000","last_change":"2023-10-06T11:22:01.632835+0000","last_active":"2023-10-06T11:22:01.642714+0000","last_peered":"2023-10-06T11:22:01.642714+0000","last_clean":"2023-10-06T11:22:01.642714+0000","last_became_active":"2023-10-06T11:22:01.632391+0000","last_became_peered":"2023-10-06T11:22:01.632391+0000","last_unstale":"2023-10-06T11:22:01.642714+0000","last_undegraded":"2023-10-06T11:22:01.642714+0000","last_fullsized":"2023-10-06T11:22:01.642714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:08.263854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611307+0000","last_change":"2023-10-06T11:21:57.561918+0000","last_active":"2023-10-06T11:22:01.611307+0000","last_peered":"2023-10-06T11:22:01.611307+0000","last_clean":"2023-10-06T11:22:01.611307+0000","last_became_active":"2023-10-06T11:21:57.560917+0000","last_became_peered":"2023-10-06T11:21:57.560917+0000","last_unstale":"2023-10-06T11:22:01.611307+0000","last_undegraded":"2023-10-06T11:22:01.611307+0000","last_fullsized":"2023-10-06T11:22:01.611307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616596+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:22:01.616596+0000","last_peered":"2023-10-06T11:22:01.616596+0000","last_clean":"2023-10-06T11:22:01.616596+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:22:01.616596+0000","last_undegraded":"2023-10-06T11:22:01.616596+0000","last_fullsized":"2023-10-06T11:22:01.616596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:02.262049+0000","last_change":"2023-10-06T11:21:56.573826+0000","last_active":"2023-10-06T11:22:02.262049+0000","last_peered":"2023-10-06T11:22:02.262049+0000","last_clean":"2023-10-06T11:22:02.262049+0000","last_became_active":"2023-10-06T11:21:56.573509+0000","last_became_peered":"2023-10-06T11:21:56.573509+0000","last_unstale":"2023-10-06T11:22:02.262049+0000","last_undegraded":"2023-10-06T11:22:02.262049+0000","last_fullsized":"2023-10-06T11:22:02.262049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634403+0000","last_change":"2023-10-06T11:22:01.631312+0000","last_active":"2023-10-06T11:22:01.634403+0000","last_peered":"2023-10-06T11:22:01.634403+0000","last_clean":"2023-10-06T11:22:01.634403+0000","last_became_active":"2023-10-06T11:22:01.630877+0000","last_became_peered":"2023-10-06T11:22:01.630877+0000","last_unstale":"2023-10-06T11:22:01.634403+0000","last_undegraded":"2023-10-06T11:22:01.634403+0000","last_fullsized":"2023-10-06T11:22:01.634403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:03:28.590751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627610+0000","last_change":"2023-10-06T11:21:57.567907+0000","last_active":"2023-10-06T11:22:01.627610+0000","last_peered":"2023-10-06T11:22:01.627610+0000","last_clean":"2023-10-06T11:22:01.627610+0000","last_became_active":"2023-10-06T11:21:57.567544+0000","last_became_peered":"2023-10-06T11:21:57.567544+0000","last_unstale":"2023-10-06T11:22:01.627610+0000","last_undegraded":"2023-10-06T11:22:01.627610+0000","last_fullsized":"2023-10-06T11:22:01.627610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.625024+0000","last_change":"2023-10-06T11:21:56.571499+0000","last_active":"2023-10-06T11:22:01.625024+0000","last_peered":"2023-10-06T11:22:01.625024+0000","last_clean":"2023-10-06T11:22:01.625024+0000","last_became_active":"2023-10-06T11:21:56.571176+0000","last_became_peered":"2023-10-06T11:21:56.571176+0000","last_unstale":"2023-10-06T11:22:01.625024+0000","last_undegraded":"2023-10-06T11:22:01.625024+0000","last_fullsized":"2023-10-06T11:22:01.625024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616847+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:22:01.616847+0000","last_peered":"2023-10-06T11:22:01.616847+0000","last_clean":"2023-10-06T11:22:01.616847+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:22:01.616847+0000","last_undegraded":"2023-10-06T11:22:01.616847+0000","last_fullsized":"2023-10-06T11:22:01.616847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614070+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:22:01.614070+0000","last_peered":"2023-10-06T11:22:01.614070+0000","last_clean":"2023-10-06T11:22:01.614070+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:22:01.614070+0000","last_undegraded":"2023-10-06T11:22:01.614070+0000","last_fullsized":"2023-10-06T11:22:01.614070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635892+0000","last_change":"2023-10-06T11:22:01.632615+0000","last_active":"2023-10-06T11:22:01.635892+0000","last_peered":"2023-10-06T11:22:01.635892+0000","last_clean":"2023-10-06T11:22:01.635892+0000","last_became_active":"2023-10-06T11:22:01.632167+0000","last_became_peered":"2023-10-06T11:22:01.632167+0000","last_unstale":"2023-10-06T11:22:01.635892+0000","last_undegraded":"2023-10-06T11:22:01.635892+0000","last_fullsized":"2023-10-06T11:22:01.635892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:35.069167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614553+0000","last_change":"2023-10-06T11:21:57.558911+0000","last_active":"2023-10-06T11:22:01.614553+0000","last_peered":"2023-10-06T11:22:01.614553+0000","last_clean":"2023-10-06T11:22:01.614553+0000","last_became_active":"2023-10-06T11:21:57.555525+0000","last_became_peered":"2023-10-06T11:21:57.555525+0000","last_unstale":"2023-10-06T11:22:01.614553+0000","last_undegraded":"2023-10-06T11:22:01.614553+0000","last_fullsized":"2023-10-06T11:22:01.614553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620129+0000","last_change":"2023-10-06T11:21:56.566441+0000","last_active":"2023-10-06T11:22:01.620129+0000","last_peered":"2023-10-06T11:22:01.620129+0000","last_clean":"2023-10-06T11:22:01.620129+0000","last_became_active":"2023-10-06T11:21:56.566070+0000","last_became_peered":"2023-10-06T11:21:56.566070+0000","last_unstale":"2023-10-06T11:22:01.620129+0000","last_undegraded":"2023-10-06T11:22:01.620129+0000","last_fullsized":"2023-10-06T11:22:01.620129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616748+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:22:01.616748+0000","last_peered":"2023-10-06T11:22:01.616748+0000","last_clean":"2023-10-06T11:22:01.616748+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:22:01.616748+0000","last_undegraded":"2023-10-06T11:22:01.616748+0000","last_fullsized":"2023-10-06T11:22:01.616748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637816+0000","last_change":"2023-10-06T11:22:01.632881+0000","last_active":"2023-10-06T11:22:01.637816+0000","last_peered":"2023-10-06T11:22:01.637816+0000","last_clean":"2023-10-06T11:22:01.637816+0000","last_became_active":"2023-10-06T11:22:01.632330+0000","last_became_peered":"2023-10-06T11:22:01.632330+0000","last_unstale":"2023-10-06T11:22:01.637816+0000","last_undegraded":"2023-10-06T11:22:01.637816+0000","last_fullsized":"2023-10-06T11:22:01.637816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:38.278034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619088+0000","last_change":"2023-10-06T11:21:57.565393+0000","last_active":"2023-10-06T11:22:01.619088+0000","last_peered":"2023-10-06T11:22:01.619088+0000","last_clean":"2023-10-06T11:22:01.619088+0000","last_became_active":"2023-10-06T11:21:57.565128+0000","last_became_peered":"2023-10-06T11:21:57.565128+0000","last_unstale":"2023-10-06T11:22:01.619088+0000","last_undegraded":"2023-10-06T11:22:01.619088+0000","last_fullsized":"2023-10-06T11:22:01.619088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611906+0000","last_change":"2023-10-06T11:21:56.573109+0000","last_active":"2023-10-06T11:22:01.611906+0000","last_peered":"2023-10-06T11:22:01.611906+0000","last_clean":"2023-10-06T11:22:01.611906+0000","last_became_active":"2023-10-06T11:21:56.572783+0000","last_became_peered":"2023-10-06T11:21:56.572783+0000","last_unstale":"2023-10-06T11:22:01.611906+0000","last_undegraded":"2023-10-06T11:22:01.611906+0000","last_fullsized":"2023-10-06T11:22:01.611906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620654+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:22:01.620654+0000","last_peered":"2023-10-06T11:22:01.620654+0000","last_clean":"2023-10-06T11:22:01.620654+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:22:01.620654+0000","last_undegraded":"2023-10-06T11:22:01.620654+0000","last_fullsized":"2023-10-06T11:22:01.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643525+0000","last_change":"2023-10-06T11:22:01.641973+0000","last_active":"2023-10-06T11:22:01.643525+0000","last_peered":"2023-10-06T11:22:01.643525+0000","last_clean":"2023-10-06T11:22:01.643525+0000","last_became_active":"2023-10-06T11:22:01.641394+0000","last_became_peered":"2023-10-06T11:22:01.641394+0000","last_unstale":"2023-10-06T11:22:01.643525+0000","last_undegraded":"2023-10-06T11:22:01.643525+0000","last_fullsized":"2023-10-06T11:22:01.643525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620183+0000","last_change":"2023-10-06T11:21:57.564937+0000","last_active":"2023-10-06T11:22:01.620183+0000","last_peered":"2023-10-06T11:22:01.620183+0000","last_clean":"2023-10-06T11:22:01.620183+0000","last_became_active":"2023-10-06T11:21:57.564603+0000","last_became_peered":"2023-10-06T11:21:57.564603+0000","last_unstale":"2023-10-06T11:22:01.620183+0000","last_undegraded":"2023-10-06T11:22:01.620183+0000","last_fullsized":"2023-10-06T11:22:01.620183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615864+0000","last_change":"2023-10-06T11:21:56.567945+0000","last_active":"2023-10-06T11:22:01.615864+0000","last_peered":"2023-10-06T11:22:01.615864+0000","last_clean":"2023-10-06T11:22:01.615864+0000","last_became_active":"2023-10-06T11:21:56.567592+0000","last_became_peered":"2023-10-06T11:21:56.567592+0000","last_unstale":"2023-10-06T11:22:01.615864+0000","last_undegraded":"2023-10-06T11:22:01.615864+0000","last_fullsized":"2023-10-06T11:22:01.615864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616944+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:22:01.616944+0000","last_peered":"2023-10-06T11:22:01.616944+0000","last_clean":"2023-10-06T11:22:01.616944+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:22:01.616944+0000","last_undegraded":"2023-10-06T11:22:01.616944+0000","last_fullsized":"2023-10-06T11:22:01.616944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644814+0000","last_change":"2023-10-06T11:22:01.639193+0000","last_active":"2023-10-06T11:22:01.644814+0000","last_peered":"2023-10-06T11:22:01.644814+0000","last_clean":"2023-10-06T11:22:01.644814+0000","last_became_active":"2023-10-06T11:22:01.638730+0000","last_became_peered":"2023-10-06T11:22:01.638730+0000","last_unstale":"2023-10-06T11:22:01.644814+0000","last_undegraded":"2023-10-06T11:22:01.644814+0000","last_fullsized":"2023-10-06T11:22:01.644814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614448+0000","last_change":"2023-10-06T11:21:57.565173+0000","last_active":"2023-10-06T11:22:01.614448+0000","last_peered":"2023-10-06T11:22:01.614448+0000","last_clean":"2023-10-06T11:22:01.614448+0000","last_became_active":"2023-10-06T11:21:57.564865+0000","last_became_peered":"2023-10-06T11:21:57.564865+0000","last_unstale":"2023-10-06T11:22:01.614448+0000","last_undegraded":"2023-10-06T11:22:01.614448+0000","last_fullsized":"2023-10-06T11:22:01.614448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616901+0000","last_change":"2023-10-06T11:21:56.570015+0000","last_active":"2023-10-06T11:22:01.616901+0000","last_peered":"2023-10-06T11:22:01.616901+0000","last_clean":"2023-10-06T11:22:01.616901+0000","last_became_active":"2023-10-06T11:21:56.569530+0000","last_became_peered":"2023-10-06T11:21:56.569530+0000","last_unstale":"2023-10-06T11:22:01.616901+0000","last_undegraded":"2023-10-06T11:22:01.616901+0000","last_fullsized":"2023-10-06T11:22:01.616901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642672+0000","last_change":"2023-10-06T11:22:01.632016+0000","last_active":"2023-10-06T11:22:01.642672+0000","last_peered":"2023-10-06T11:22:01.642672+0000","last_clean":"2023-10-06T11:22:01.642672+0000","last_became_active":"2023-10-06T11:22:01.629544+0000","last_became_peered":"2023-10-06T11:22:01.629544+0000","last_unstale":"2023-10-06T11:22:01.642672+0000","last_undegraded":"2023-10-06T11:22:01.642672+0000","last_fullsized":"2023-10-06T11:22:01.642672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:20.535717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611336+0000","last_change":"2023-10-06T11:21:57.566827+0000","last_active":"2023-10-06T11:22:01.611336+0000","last_peered":"2023-10-06T11:22:01.611336+0000","last_clean":"2023-10-06T11:22:01.611336+0000","last_became_active":"2023-10-06T11:21:57.566463+0000","last_became_peered":"2023-10-06T11:21:57.566463+0000","last_unstale":"2023-10-06T11:22:01.611336+0000","last_undegraded":"2023-10-06T11:22:01.611336+0000","last_fullsized":"2023-10-06T11:22:01.611336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611609+0000","last_change":"2023-10-06T11:21:56.573082+0000","last_active":"2023-10-06T11:22:01.611609+0000","last_peered":"2023-10-06T11:22:01.611609+0000","last_clean":"2023-10-06T11:22:01.611609+0000","last_became_active":"2023-10-06T11:21:56.572363+0000","last_became_peered":"2023-10-06T11:21:56.572363+0000","last_unstale":"2023-10-06T11:22:01.611609+0000","last_undegraded":"2023-10-06T11:22:01.611609+0000","last_fullsized":"2023-10-06T11:22:01.611609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.628822+0000","last_change":"2023-10-06T11:22:01.626277+0000","last_active":"2023-10-06T11:22:01.628822+0000","last_peered":"2023-10-06T11:22:01.628822+0000","last_clean":"2023-10-06T11:22:01.628822+0000","last_became_active":"2023-10-06T11:22:01.625806+0000","last_became_peered":"2023-10-06T11:22:01.625806+0000","last_unstale":"2023-10-06T11:22:01.628822+0000","last_undegraded":"2023-10-06T11:22:01.628822+0000","last_fullsized":"2023-10-06T11:22:01.628822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611575+0000","last_change":"2023-10-06T11:21:57.564883+0000","last_active":"2023-10-06T11:22:01.611575+0000","last_peered":"2023-10-06T11:22:01.611575+0000","last_clean":"2023-10-06T11:22:01.611575+0000","last_became_active":"2023-10-06T11:21:57.564586+0000","last_became_peered":"2023-10-06T11:21:57.564586+0000","last_unstale":"2023-10-06T11:22:01.611575+0000","last_undegraded":"2023-10-06T11:22:01.611575+0000","last_fullsized":"2023-10-06T11:22:01.611575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:12.024410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621041+0000","last_change":"2023-10-06T11:21:56.576790+0000","last_active":"2023-10-06T11:22:01.621041+0000","last_peered":"2023-10-06T11:22:01.621041+0000","last_clean":"2023-10-06T11:22:01.621041+0000","last_became_active":"2023-10-06T11:21:56.576463+0000","last_became_peered":"2023-10-06T11:21:56.576463+0000","last_unstale":"2023-10-06T11:22:01.621041+0000","last_undegraded":"2023-10-06T11:22:01.621041+0000","last_fullsized":"2023-10-06T11:22:01.621041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643771+0000","last_change":"2023-10-06T11:22:01.639619+0000","last_active":"2023-10-06T11:22:01.643771+0000","last_peered":"2023-10-06T11:22:01.643771+0000","last_clean":"2023-10-06T11:22:01.643771+0000","last_became_active":"2023-10-06T11:22:01.638615+0000","last_became_peered":"2023-10-06T11:22:01.638615+0000","last_unstale":"2023-10-06T11:22:01.643771+0000","last_undegraded":"2023-10-06T11:22:01.643771+0000","last_fullsized":"2023-10-06T11:22:01.643771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612709+0000","last_change":"2023-10-06T11:21:57.577150+0000","last_active":"2023-10-06T11:22:01.612709+0000","last_peered":"2023-10-06T11:22:01.612709+0000","last_clean":"2023-10-06T11:22:01.612709+0000","last_became_active":"2023-10-06T11:21:57.576632+0000","last_became_peered":"2023-10-06T11:21:57.576632+0000","last_unstale":"2023-10-06T11:22:01.612709+0000","last_undegraded":"2023-10-06T11:22:01.612709+0000","last_fullsized":"2023-10-06T11:22:01.612709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:32.416966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617372+0000","last_change":"2023-10-06T11:21:56.575818+0000","last_active":"2023-10-06T11:22:01.617372+0000","last_peered":"2023-10-06T11:22:01.617372+0000","last_clean":"2023-10-06T11:22:01.617372+0000","last_became_active":"2023-10-06T11:21:56.575460+0000","last_became_peered":"2023-10-06T11:21:56.575460+0000","last_unstale":"2023-10-06T11:22:01.617372+0000","last_undegraded":"2023-10-06T11:22:01.617372+0000","last_fullsized":"2023-10-06T11:22:01.617372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634446+0000","last_change":"2023-10-06T11:22:01.632266+0000","last_active":"2023-10-06T11:22:01.634446+0000","last_peered":"2023-10-06T11:22:01.634446+0000","last_clean":"2023-10-06T11:22:01.634446+0000","last_became_active":"2023-10-06T11:22:01.631851+0000","last_became_peered":"2023-10-06T11:22:01.631851+0000","last_unstale":"2023-10-06T11:22:01.634446+0000","last_undegraded":"2023-10-06T11:22:01.634446+0000","last_fullsized":"2023-10-06T11:22:01.634446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:11.811755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626480+0000","last_change":"2023-10-06T11:21:57.561599+0000","last_active":"2023-10-06T11:22:01.626480+0000","last_peered":"2023-10-06T11:22:01.626480+0000","last_clean":"2023-10-06T11:22:01.626480+0000","last_became_active":"2023-10-06T11:21:57.561280+0000","last_became_peered":"2023-10-06T11:21:57.561280+0000","last_unstale":"2023-10-06T11:22:01.626480+0000","last_undegraded":"2023-10-06T11:22:01.626480+0000","last_fullsized":"2023-10-06T11:22:01.626480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621119+0000","last_change":"2023-10-06T11:21:56.566817+0000","last_active":"2023-10-06T11:22:01.621119+0000","last_peered":"2023-10-06T11:22:01.621119+0000","last_clean":"2023-10-06T11:22:01.621119+0000","last_became_active":"2023-10-06T11:21:56.566466+0000","last_became_peered":"2023-10-06T11:21:56.566466+0000","last_unstale":"2023-10-06T11:22:01.621119+0000","last_undegraded":"2023-10-06T11:22:01.621119+0000","last_fullsized":"2023-10-06T11:22:01.621119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.632874+0000","last_change":"2023-10-06T11:22:01.628287+0000","last_active":"2023-10-06T11:22:01.632874+0000","last_peered":"2023-10-06T11:22:01.632874+0000","last_clean":"2023-10-06T11:22:01.632874+0000","last_became_active":"2023-10-06T11:22:01.627842+0000","last_became_peered":"2023-10-06T11:22:01.627842+0000","last_unstale":"2023-10-06T11:22:01.632874+0000","last_undegraded":"2023-10-06T11:22:01.632874+0000","last_fullsized":"2023-10-06T11:22:01.632874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:42.027969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620876+0000","last_change":"2023-10-06T11:21:57.572846+0000","last_active":"2023-10-06T11:22:01.620876+0000","last_peered":"2023-10-06T11:22:01.620876+0000","last_clean":"2023-10-06T11:22:01.620876+0000","last_became_active":"2023-10-06T11:21:57.572567+0000","last_became_peered":"2023-10-06T11:21:57.572567+0000","last_unstale":"2023-10-06T11:22:01.620876+0000","last_undegraded":"2023-10-06T11:22:01.620876+0000","last_fullsized":"2023-10-06T11:22:01.620876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620369+0000","last_change":"2023-10-06T11:21:56.575220+0000","last_active":"2023-10-06T11:22:01.620369+0000","last_peered":"2023-10-06T11:22:01.620369+0000","last_clean":"2023-10-06T11:22:01.620369+0000","last_became_active":"2023-10-06T11:21:56.574557+0000","last_became_peered":"2023-10-06T11:21:56.574557+0000","last_unstale":"2023-10-06T11:22:01.620369+0000","last_undegraded":"2023-10-06T11:22:01.620369+0000","last_fullsized":"2023-10-06T11:22:01.620369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639894+0000","last_change":"2023-10-06T11:22:01.637484+0000","last_active":"2023-10-06T11:22:01.639894+0000","last_peered":"2023-10-06T11:22:01.639894+0000","last_clean":"2023-10-06T11:22:01.639894+0000","last_became_active":"2023-10-06T11:22:01.637045+0000","last_became_peered":"2023-10-06T11:22:01.637045+0000","last_unstale":"2023-10-06T11:22:01.639894+0000","last_undegraded":"2023-10-06T11:22:01.639894+0000","last_fullsized":"2023-10-06T11:22:01.639894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:36.472691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610908+0000","last_change":"2023-10-06T11:21:57.562666+0000","last_active":"2023-10-06T11:22:01.610908+0000","last_peered":"2023-10-06T11:22:01.610908+0000","last_clean":"2023-10-06T11:22:01.610908+0000","last_became_active":"2023-10-06T11:21:57.562384+0000","last_became_peered":"2023-10-06T11:21:57.562384+0000","last_unstale":"2023-10-06T11:22:01.610908+0000","last_undegraded":"2023-10-06T11:22:01.610908+0000","last_fullsized":"2023-10-06T11:22:01.610908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:03.832047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619654+0000","last_change":"2023-10-06T11:21:56.578885+0000","last_active":"2023-10-06T11:22:01.619654+0000","last_peered":"2023-10-06T11:22:01.619654+0000","last_clean":"2023-10-06T11:22:01.619654+0000","last_became_active":"2023-10-06T11:21:56.578589+0000","last_became_peered":"2023-10-06T11:21:56.578589+0000","last_unstale":"2023-10-06T11:22:01.619654+0000","last_undegraded":"2023-10-06T11:22:01.619654+0000","last_fullsized":"2023-10-06T11:22:01.619654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:31.263377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.629203+0000","last_change":"2023-10-06T11:22:01.626674+0000","last_active":"2023-10-06T11:22:01.629203+0000","last_peered":"2023-10-06T11:22:01.629203+0000","last_clean":"2023-10-06T11:22:01.629203+0000","last_became_active":"2023-10-06T11:22:01.626199+0000","last_became_peered":"2023-10-06T11:22:01.626199+0000","last_unstale":"2023-10-06T11:22:01.629203+0000","last_undegraded":"2023-10-06T11:22:01.629203+0000","last_fullsized":"2023-10-06T11:22:01.629203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:07.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619746+0000","last_change":"2023-10-06T11:21:57.568271+0000","last_active":"2023-10-06T11:22:01.619746+0000","last_peered":"2023-10-06T11:22:01.619746+0000","last_clean":"2023-10-06T11:22:01.619746+0000","last_became_active":"2023-10-06T11:21:57.567943+0000","last_became_peered":"2023-10-06T11:21:57.567943+0000","last_unstale":"2023-10-06T11:22:01.619746+0000","last_undegraded":"2023-10-06T11:22:01.619746+0000","last_fullsized":"2023-10-06T11:22:01.619746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619520+0000","last_change":"2023-10-06T11:21:56.579019+0000","last_active":"2023-10-06T11:22:01.619520+0000","last_peered":"2023-10-06T11:22:01.619520+0000","last_clean":"2023-10-06T11:22:01.619520+0000","last_became_active":"2023-10-06T11:21:56.578659+0000","last_became_peered":"2023-10-06T11:21:56.578659+0000","last_unstale":"2023-10-06T11:22:01.619520+0000","last_undegraded":"2023-10-06T11:22:01.619520+0000","last_fullsized":"2023-10-06T11:22:01.619520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643049+0000","last_change":"2023-10-06T11:22:01.637818+0000","last_active":"2023-10-06T11:22:01.643049+0000","last_peered":"2023-10-06T11:22:01.643049+0000","last_clean":"2023-10-06T11:22:01.643049+0000","last_became_active":"2023-10-06T11:22:01.637325+0000","last_became_peered":"2023-10-06T11:22:01.637325+0000","last_unstale":"2023-10-06T11:22:01.643049+0000","last_undegraded":"2023-10-06T11:22:01.643049+0000","last_fullsized":"2023-10-06T11:22:01.643049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641080+0000","last_change":"2023-10-06T11:22:01.636779+0000","last_active":"2023-10-06T11:22:01.641080+0000","last_peered":"2023-10-06T11:22:01.641080+0000","last_clean":"2023-10-06T11:22:01.641080+0000","last_became_active":"2023-10-06T11:22:01.636344+0000","last_became_peered":"2023-10-06T11:22:01.636344+0000","last_unstale":"2023-10-06T11:22:01.641080+0000","last_undegraded":"2023-10-06T11:22:01.641080+0000","last_fullsized":"2023-10-06T11:22:01.641080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620730+0000","last_change":"2023-10-06T11:21:56.575584+0000","last_active":"2023-10-06T11:22:01.620730+0000","last_peered":"2023-10-06T11:22:01.620730+0000","last_clean":"2023-10-06T11:22:01.620730+0000","last_became_active":"2023-10-06T11:21:56.575201+0000","last_became_peered":"2023-10-06T11:21:56.575201+0000","last_unstale":"2023-10-06T11:22:01.620730+0000","last_undegraded":"2023-10-06T11:22:01.620730+0000","last_fullsized":"2023-10-06T11:22:01.620730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616376+0000","last_change":"2023-10-06T11:21:57.568259+0000","last_active":"2023-10-06T11:22:01.616376+0000","last_peered":"2023-10-06T11:22:01.616376+0000","last_clean":"2023-10-06T11:22:01.616376+0000","last_became_active":"2023-10-06T11:21:57.567958+0000","last_became_peered":"2023-10-06T11:21:57.567958+0000","last_unstale":"2023-10-06T11:22:01.616376+0000","last_undegraded":"2023-10-06T11:22:01.616376+0000","last_fullsized":"2023-10-06T11:22:01.616376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637658+0000","last_change":"2023-10-06T11:22:01.635766+0000","last_active":"2023-10-06T11:22:01.637658+0000","last_peered":"2023-10-06T11:22:01.637658+0000","last_clean":"2023-10-06T11:22:01.637658+0000","last_became_active":"2023-10-06T11:22:01.635314+0000","last_became_peered":"2023-10-06T11:22:01.635314+0000","last_unstale":"2023-10-06T11:22:01.637658+0000","last_undegraded":"2023-10-06T11:22:01.637658+0000","last_fullsized":"2023-10-06T11:22:01.637658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:33:19.459200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619865+0000","last_change":"2023-10-06T11:21:56.579444+0000","last_active":"2023-10-06T11:22:01.619865+0000","last_peered":"2023-10-06T11:22:01.619865+0000","last_clean":"2023-10-06T11:22:01.619865+0000","last_became_active":"2023-10-06T11:21:56.578427+0000","last_became_peered":"2023-10-06T11:21:56.578427+0000","last_unstale":"2023-10-06T11:22:01.619865+0000","last_undegraded":"2023-10-06T11:22:01.619865+0000","last_fullsized":"2023-10-06T11:22:01.619865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:21.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627696+0000","last_change":"2023-10-06T11:21:57.569150+0000","last_active":"2023-10-06T11:22:01.627696+0000","last_peered":"2023-10-06T11:22:01.627696+0000","last_clean":"2023-10-06T11:22:01.627696+0000","last_became_active":"2023-10-06T11:21:57.568883+0000","last_became_peered":"2023-10-06T11:21:57.568883+0000","last_unstale":"2023-10-06T11:22:01.627696+0000","last_undegraded":"2023-10-06T11:22:01.627696+0000","last_fullsized":"2023-10-06T11:22:01.627696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626255+0000","last_change":"2023-10-06T11:21:56.572563+0000","last_active":"2023-10-06T11:22:01.626255+0000","last_peered":"2023-10-06T11:22:01.626255+0000","last_clean":"2023-10-06T11:22:01.626255+0000","last_became_active":"2023-10-06T11:21:56.571525+0000","last_became_peered":"2023-10-06T11:21:56.571525+0000","last_unstale":"2023-10-06T11:22:01.626255+0000","last_undegraded":"2023-10-06T11:22:01.626255+0000","last_fullsized":"2023-10-06T11:22:01.626255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621535+0000","last_change":"2023-10-06T11:21:57.569979+0000","last_active":"2023-10-06T11:22:01.621535+0000","last_peered":"2023-10-06T11:22:01.621535+0000","last_clean":"2023-10-06T11:22:01.621535+0000","last_became_active":"2023-10-06T11:21:57.569669+0000","last_became_peered":"2023-10-06T11:21:57.569669+0000","last_unstale":"2023-10-06T11:22:01.621535+0000","last_undegraded":"2023-10-06T11:22:01.621535+0000","last_fullsized":"2023-10-06T11:22:01.621535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.638165+0000","last_change":"2023-10-06T11:22:01.636951+0000","last_active":"2023-10-06T11:22:01.638165+0000","last_peered":"2023-10-06T11:22:01.638165+0000","last_clean":"2023-10-06T11:22:01.638165+0000","last_became_active":"2023-10-06T11:22:01.636534+0000","last_became_peered":"2023-10-06T11:22:01.636534+0000","last_unstale":"2023-10-06T11:22:01.638165+0000","last_undegraded":"2023-10-06T11:22:01.638165+0000","last_fullsized":"2023-10-06T11:22:01.638165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621855+0000","last_change":"2023-10-06T11:21:56.579219+0000","last_active":"2023-10-06T11:22:01.621855+0000","last_peered":"2023-10-06T11:22:01.621855+0000","last_clean":"2023-10-06T11:22:01.621855+0000","last_became_active":"2023-10-06T11:21:56.578100+0000","last_became_peered":"2023-10-06T11:21:56.578100+0000","last_unstale":"2023-10-06T11:22:01.621855+0000","last_undegraded":"2023-10-06T11:22:01.621855+0000","last_fullsized":"2023-10-06T11:22:01.621855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616499+0000","last_change":"2023-10-06T11:21:57.580696+0000","last_active":"2023-10-06T11:22:01.616499+0000","last_peered":"2023-10-06T11:22:01.616499+0000","last_clean":"2023-10-06T11:22:01.616499+0000","last_became_active":"2023-10-06T11:21:57.580262+0000","last_became_peered":"2023-10-06T11:21:57.580262+0000","last_unstale":"2023-10-06T11:22:01.616499+0000","last_undegraded":"2023-10-06T11:22:01.616499+0000","last_fullsized":"2023-10-06T11:22:01.616499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635928+0000","last_change":"2023-10-06T11:22:01.632977+0000","last_active":"2023-10-06T11:22:01.635928+0000","last_peered":"2023-10-06T11:22:01.635928+0000","last_clean":"2023-10-06T11:22:01.635928+0000","last_became_active":"2023-10-06T11:22:01.632522+0000","last_became_peered":"2023-10-06T11:22:01.632522+0000","last_unstale":"2023-10-06T11:22:01.635928+0000","last_undegraded":"2023-10-06T11:22:01.635928+0000","last_fullsized":"2023-10-06T11:22:01.635928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620591+0000","last_change":"2023-10-06T11:21:57.570931+0000","last_active":"2023-10-06T11:22:01.620591+0000","last_peered":"2023-10-06T11:22:01.620591+0000","last_clean":"2023-10-06T11:22:01.620591+0000","last_became_active":"2023-10-06T11:21:57.570619+0000","last_became_peered":"2023-10-06T11:21:57.570619+0000","last_unstale":"2023-10-06T11:22:01.620591+0000","last_undegraded":"2023-10-06T11:22:01.620591+0000","last_fullsized":"2023-10-06T11:22:01.620591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615615+0000","last_change":"2023-10-06T11:21:57.572767+0000","last_active":"2023-10-06T11:22:01.615615+0000","last_peered":"2023-10-06T11:22:01.615615+0000","last_clean":"2023-10-06T11:22:01.615615+0000","last_became_active":"2023-10-06T11:21:57.572371+0000","last_became_peered":"2023-10-06T11:21:57.572371+0000","last_unstale":"2023-10-06T11:22:01.615615+0000","last_undegraded":"2023-10-06T11:22:01.615615+0000","last_fullsized":"2023-10-06T11:22:01.615615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610290+0000","last_change":"2023-10-06T11:21:57.571303+0000","last_active":"2023-10-06T11:22:01.610290+0000","last_peered":"2023-10-06T11:22:01.610290+0000","last_clean":"2023-10-06T11:22:01.610290+0000","last_became_active":"2023-10-06T11:21:57.570978+0000","last_became_peered":"2023-10-06T11:21:57.570978+0000","last_unstale":"2023-10-06T11:22:01.610290+0000","last_undegraded":"2023-10-06T11:22:01.610290+0000","last_fullsized":"2023-10-06T11:22:01.610290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617222+0000","last_change":"2023-10-06T11:21:57.569031+0000","last_active":"2023-10-06T11:22:01.617222+0000","last_peered":"2023-10-06T11:22:01.617222+0000","last_clean":"2023-10-06T11:22:01.617222+0000","last_became_active":"2023-10-06T11:21:57.568734+0000","last_became_peered":"2023-10-06T11:21:57.568734+0000","last_unstale":"2023-10-06T11:22:01.617222+0000","last_undegraded":"2023-10-06T11:22:01.617222+0000","last_fullsized":"2023-10-06T11:22:01.617222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619002+0000","last_change":"2023-10-06T11:21:57.579632+0000","last_active":"2023-10-06T11:22:01.619002+0000","last_peered":"2023-10-06T11:22:01.619002+0000","last_clean":"2023-10-06T11:22:01.619002+0000","last_became_active":"2023-10-06T11:21:57.567143+0000","last_became_peered":"2023-10-06T11:21:57.567143+0000","last_unstale":"2023-10-06T11:22:01.619002+0000","last_undegraded":"2023-10-06T11:22:01.619002+0000","last_fullsized":"2023-10-06T11:22:01.619002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621442+0000","last_change":"2023-10-06T11:21:57.569847+0000","last_active":"2023-10-06T11:22:01.621442+0000","last_peered":"2023-10-06T11:22:01.621442+0000","last_clean":"2023-10-06T11:22:01.621442+0000","last_became_active":"2023-10-06T11:21:57.569571+0000","last_became_peered":"2023-10-06T11:21:57.569571+0000","last_unstale":"2023-10-06T11:22:01.621442+0000","last_undegraded":"2023-10-06T11:22:01.621442+0000","last_fullsized":"2023-10-06T11:22:01.621442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621862+0000","last_change":"2023-10-06T11:21:57.562969+0000","last_active":"2023-10-06T11:22:01.621862+0000","last_peered":"2023-10-06T11:22:01.621862+0000","last_clean":"2023-10-06T11:22:01.621862+0000","last_became_active":"2023-10-06T11:21:57.562623+0000","last_became_peered":"2023-10-06T11:21:57.562623+0000","last_unstale":"2023-10-06T11:22:01.621862+0000","last_undegraded":"2023-10-06T11:22:01.621862+0000","last_fullsized":"2023-10-06T11:22:01.621862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614335+0000","last_change":"2023-10-06T11:21:57.571639+0000","last_active":"2023-10-06T11:22:01.614335+0000","last_peered":"2023-10-06T11:22:01.614335+0000","last_clean":"2023-10-06T11:22:01.614335+0000","last_became_active":"2023-10-06T11:21:57.571347+0000","last_became_peered":"2023-10-06T11:21:57.571347+0000","last_unstale":"2023-10-06T11:22:01.614335+0000","last_undegraded":"2023-10-06T11:22:01.614335+0000","last_fullsized":"2023-10-06T11:22:01.614335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616721+0000","last_change":"2023-10-06T11:21:57.576904+0000","last_active":"2023-10-06T11:22:01.616721+0000","last_peered":"2023-10-06T11:22:01.616721+0000","last_clean":"2023-10-06T11:22:01.616721+0000","last_became_active":"2023-10-06T11:21:57.576570+0000","last_became_peered":"2023-10-06T11:21:57.576570+0000","last_unstale":"2023-10-06T11:22:01.616721+0000","last_undegraded":"2023-10-06T11:22:01.616721+0000","last_fullsized":"2023-10-06T11:22:01.616721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615508+0000","last_change":"2023-10-06T11:21:57.572666+0000","last_active":"2023-10-06T11:22:01.615508+0000","last_peered":"2023-10-06T11:22:01.615508+0000","last_clean":"2023-10-06T11:22:01.615508+0000","last_became_active":"2023-10-06T11:21:57.569109+0000","last_became_peered":"2023-10-06T11:21:57.569109+0000","last_unstale":"2023-10-06T11:22:01.615508+0000","last_undegraded":"2023-10-06T11:22:01.615508+0000","last_fullsized":"2023-10-06T11:22:01.615508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626091+0000","last_change":"2023-10-06T11:21:57.566406+0000","last_active":"2023-10-06T11:22:01.626091+0000","last_peered":"2023-10-06T11:22:01.626091+0000","last_clean":"2023-10-06T11:22:01.626091+0000","last_became_active":"2023-10-06T11:21:57.565954+0000","last_became_peered":"2023-10-06T11:21:57.565954+0000","last_unstale":"2023-10-06T11:22:01.626091+0000","last_undegraded":"2023-10-06T11:22:01.626091+0000","last_fullsized":"2023-10-06T11:22:01.626091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621753+0000","last_change":"2023-10-06T11:21:57.576914+0000","last_active":"2023-10-06T11:22:01.621753+0000","last_peered":"2023-10-06T11:22:01.621753+0000","last_clean":"2023-10-06T11:22:01.621753+0000","last_became_active":"2023-10-06T11:21:57.576444+0000","last_became_peered":"2023-10-06T11:21:57.576444+0000","last_unstale":"2023-10-06T11:22:01.621753+0000","last_undegraded":"2023-10-06T11:22:01.621753+0000","last_fullsized":"2023-10-06T11:22:01.621753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612641+0000","last_change":"2023-10-06T11:21:57.565842+0000","last_active":"2023-10-06T11:22:01.612641+0000","last_peered":"2023-10-06T11:22:01.612641+0000","last_clean":"2023-10-06T11:22:01.612641+0000","last_became_active":"2023-10-06T11:21:57.565455+0000","last_became_peered":"2023-10-06T11:21:57.565455+0000","last_unstale":"2023-10-06T11:22:01.612641+0000","last_undegraded":"2023-10-06T11:22:01.612641+0000","last_fullsized":"2023-10-06T11:22:01.612641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:28.522557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612331+0000","last_change":"2023-10-06T11:21:57.569901+0000","last_active":"2023-10-06T11:22:01.612331+0000","last_peered":"2023-10-06T11:22:01.612331+0000","last_clean":"2023-10-06T11:22:01.612331+0000","last_became_active":"2023-10-06T11:21:57.569642+0000","last_became_peered":"2023-10-06T11:21:57.569642+0000","last_unstale":"2023-10-06T11:22:01.612331+0000","last_undegraded":"2023-10-06T11:22:01.612331+0000","last_fullsized":"2023-10-06T11:22:01.612331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.950303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611118+0000","last_change":"2023-10-06T11:21:57.564154+0000","last_active":"2023-10-06T11:22:01.611118+0000","last_peered":"2023-10-06T11:22:01.611118+0000","last_clean":"2023-10-06T11:22:01.611118+0000","last_became_active":"2023-10-06T11:21:57.563840+0000","last_became_peered":"2023-10-06T11:21:57.563840+0000","last_unstale":"2023-10-06T11:22:01.611118+0000","last_undegraded":"2023-10-06T11:22:01.611118+0000","last_fullsized":"2023-10-06T11:22:01.611118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627637+0000","last_change":"2023-10-06T11:21:57.569223+0000","last_active":"2023-10-06T11:22:01.627637+0000","last_peered":"2023-10-06T11:22:01.627637+0000","last_clean":"2023-10-06T11:22:01.627637+0000","last_became_active":"2023-10-06T11:21:57.568967+0000","last_became_peered":"2023-10-06T11:21:57.568967+0000","last_unstale":"2023-10-06T11:22:01.627637+0000","last_undegraded":"2023-10-06T11:22:01.627637+0000","last_fullsized":"2023-10-06T11:22:01.627637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612585+0000","last_change":"2023-10-06T11:21:57.567578+0000","last_active":"2023-10-06T11:22:01.612585+0000","last_peered":"2023-10-06T11:22:01.612585+0000","last_clean":"2023-10-06T11:22:01.612585+0000","last_became_active":"2023-10-06T11:21:57.565977+0000","last_became_peered":"2023-10-06T11:21:57.565977+0000","last_unstale":"2023-10-06T11:22:01.612585+0000","last_undegraded":"2023-10-06T11:22:01.612585+0000","last_fullsized":"2023-10-06T11:22:01.612585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619766+0000","last_change":"2023-10-06T11:21:57.576090+0000","last_active":"2023-10-06T11:22:01.619766+0000","last_peered":"2023-10-06T11:22:01.619766+0000","last_clean":"2023-10-06T11:22:01.619766+0000","last_became_active":"2023-10-06T11:21:57.575667+0000","last_became_peered":"2023-10-06T11:21:57.575667+0000","last_unstale":"2023-10-06T11:22:01.619766+0000","last_undegraded":"2023-10-06T11:22:01.619766+0000","last_fullsized":"2023-10-06T11:22:01.619766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621178+0000","last_change":"2023-10-06T11:21:57.563777+0000","last_active":"2023-10-06T11:22:01.621178+0000","last_peered":"2023-10-06T11:22:01.621178+0000","last_clean":"2023-10-06T11:22:01.621178+0000","last_became_active":"2023-10-06T11:21:57.563485+0000","last_became_peered":"2023-10-06T11:21:57.563485+0000","last_unstale":"2023-10-06T11:22:01.621178+0000","last_undegraded":"2023-10-06T11:22:01.621178+0000","last_fullsized":"2023-10-06T11:22:01.621178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610375+0000","last_change":"2023-10-06T11:21:57.575519+0000","last_active":"2023-10-06T11:22:01.610375+0000","last_peered":"2023-10-06T11:22:01.610375+0000","last_clean":"2023-10-06T11:22:01.610375+0000","last_became_active":"2023-10-06T11:21:57.575122+0000","last_became_peered":"2023-10-06T11:21:57.575122+0000","last_unstale":"2023-10-06T11:22:01.610375+0000","last_undegraded":"2023-10-06T11:22:01.610375+0000","last_fullsized":"2023-10-06T11:22:01.610375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:01.582258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607465+0000","last_change":"2023-10-06T11:21:57.563806+0000","last_active":"2023-10-06T11:22:01.607465+0000","last_peered":"2023-10-06T11:22:01.607465+0000","last_clean":"2023-10-06T11:22:01.607465+0000","last_became_active":"2023-10-06T11:21:57.561796+0000","last_became_peered":"2023-10-06T11:21:57.561796+0000","last_unstale":"2023-10-06T11:22:01.607465+0000","last_undegraded":"2023-10-06T11:22:01.607465+0000","last_fullsized":"2023-10-06T11:22:01.607465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610200+0000","last_change":"2023-10-06T11:21:57.560661+0000","last_active":"2023-10-06T11:22:01.610200+0000","last_peered":"2023-10-06T11:22:01.610200+0000","last_clean":"2023-10-06T11:22:01.610200+0000","last_became_active":"2023-10-06T11:21:57.559848+0000","last_became_peered":"2023-10-06T11:21:57.559848+0000","last_unstale":"2023-10-06T11:22:01.610200+0000","last_undegraded":"2023-10-06T11:22:01.610200+0000","last_fullsized":"2023-10-06T11:22:01.610200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621548+0000","last_change":"2023-10-06T11:21:57.566423+0000","last_active":"2023-10-06T11:22:01.621548+0000","last_peered":"2023-10-06T11:22:01.621548+0000","last_clean":"2023-10-06T11:22:01.621548+0000","last_became_active":"2023-10-06T11:21:57.566064+0000","last_became_peered":"2023-10-06T11:21:57.566064+0000","last_unstale":"2023-10-06T11:22:01.621548+0000","last_undegraded":"2023-10-06T11:22:01.621548+0000","last_fullsized":"2023-10-06T11:22:01.621548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627537+0000","last_change":"2023-10-06T11:21:57.575695+0000","last_active":"2023-10-06T11:22:01.627537+0000","last_peered":"2023-10-06T11:22:01.627537+0000","last_clean":"2023-10-06T11:22:01.627537+0000","last_became_active":"2023-10-06T11:21:57.575418+0000","last_became_peered":"2023-10-06T11:21:57.575418+0000","last_unstale":"2023-10-06T11:22:01.627537+0000","last_undegraded":"2023-10-06T11:22:01.627537+0000","last_fullsized":"2023-10-06T11:22:01.627537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619861+0000","last_change":"2023-10-06T11:21:57.562165+0000","last_active":"2023-10-06T11:22:01.619861+0000","last_peered":"2023-10-06T11:22:01.619861+0000","last_clean":"2023-10-06T11:22:01.619861+0000","last_became_active":"2023-10-06T11:21:57.561878+0000","last_became_peered":"2023-10-06T11:21:57.561878+0000","last_unstale":"2023-10-06T11:22:01.619861+0000","last_undegraded":"2023-10-06T11:22:01.619861+0000","last_fullsized":"2023-10-06T11:22:01.619861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619664+0000","last_change":"2023-10-06T11:21:57.575950+0000","last_active":"2023-10-06T11:22:01.619664+0000","last_peered":"2023-10-06T11:22:01.619664+0000","last_clean":"2023-10-06T11:22:01.619664+0000","last_became_active":"2023-10-06T11:21:57.575280+0000","last_became_peered":"2023-10-06T11:21:57.575280+0000","last_unstale":"2023-10-06T11:22:01.619664+0000","last_undegraded":"2023-10-06T11:22:01.619664+0000","last_fullsized":"2023-10-06T11:22:01.619664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609933+0000","last_change":"2023-10-06T11:21:57.566684+0000","last_active":"2023-10-06T11:22:01.609933+0000","last_peered":"2023-10-06T11:22:01.609933+0000","last_clean":"2023-10-06T11:22:01.609933+0000","last_became_active":"2023-10-06T11:21:57.566358+0000","last_became_peered":"2023-10-06T11:21:57.566358+0000","last_unstale":"2023-10-06T11:22:01.609933+0000","last_undegraded":"2023-10-06T11:22:01.609933+0000","last_fullsized":"2023-10-06T11:22:01.609933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:50.324902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626719+0000","last_change":"2023-10-06T11:21:57.574099+0000","last_active":"2023-10-06T11:22:01.626719+0000","last_peered":"2023-10-06T11:22:01.626719+0000","last_clean":"2023-10-06T11:22:01.626719+0000","last_became_active":"2023-10-06T11:21:57.573592+0000","last_became_peered":"2023-10-06T11:21:57.573592+0000","last_unstale":"2023-10-06T11:22:01.626719+0000","last_undegraded":"2023-10-06T11:22:01.626719+0000","last_fullsized":"2023-10-06T11:22:01.626719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619959+0000","last_change":"2023-10-06T11:21:57.566310+0000","last_active":"2023-10-06T11:22:01.619959+0000","last_peered":"2023-10-06T11:22:01.619959+0000","last_clean":"2023-10-06T11:22:01.619959+0000","last_became_active":"2023-10-06T11:21:57.566014+0000","last_became_peered":"2023-10-06T11:21:57.566014+0000","last_unstale":"2023-10-06T11:22:01.619959+0000","last_undegraded":"2023-10-06T11:22:01.619959+0000","last_fullsized":"2023-10-06T11:22:01.619959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.268101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609857+0000","last_change":"2023-10-06T11:21:57.567982+0000","last_active":"2023-10-06T11:22:01.609857+0000","last_peered":"2023-10-06T11:22:01.609857+0000","last_clean":"2023-10-06T11:22:01.609857+0000","last_became_active":"2023-10-06T11:21:57.567667+0000","last_became_peered":"2023-10-06T11:21:57.567667+0000","last_unstale":"2023-10-06T11:22:01.609857+0000","last_undegraded":"2023-10-06T11:22:01.609857+0000","last_fullsized":"2023-10-06T11:22:01.609857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:14.276554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621434+0000","last_change":"2023-10-06T11:21:57.562657+0000","last_active":"2023-10-06T11:22:01.621434+0000","last_peered":"2023-10-06T11:22:01.621434+0000","last_clean":"2023-10-06T11:22:01.621434+0000","last_became_active":"2023-10-06T11:21:57.562338+0000","last_became_peered":"2023-10-06T11:21:57.562338+0000","last_unstale":"2023-10-06T11:22:01.621434+0000","last_undegraded":"2023-10-06T11:22:01.621434+0000","last_fullsized":"2023-10-06T11:22:01.621434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607967+0000","last_change":"2023-10-06T11:21:57.569451+0000","last_active":"2023-10-06T11:22:01.607967+0000","last_peered":"2023-10-06T11:22:01.607967+0000","last_clean":"2023-10-06T11:22:01.607967+0000","last_became_active":"2023-10-06T11:21:57.569124+0000","last_became_peered":"2023-10-06T11:21:57.569124+0000","last_unstale":"2023-10-06T11:22:01.607967+0000","last_undegraded":"2023-10-06T11:22:01.607967+0000","last_fullsized":"2023-10-06T11:22:01.607967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.833392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":703756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301652,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070188,"statfs":{"total":96636764160,"available":96327872512,"internally_reserved":0,"allocated":274432,"data_stored":110082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":286720,"data_stored":113228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":109548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302224,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069616,"statfs":{"total":96636764160,"available":96327286784,"internally_reserved":0,"allocated":860160,"data_stored":699826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":110012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:22:17.600 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:22:17.601 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T11:22:17.601 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T11:22:17.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T11:22:17.601 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T11:22:17.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 Processor -- start 2023-10-06T11:22:17.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 -- start start 2023-10-06T11:22:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb8c158e60 0x7fbb8c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c068540 0x7fbb8c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c0c95f0 con 0x7fbb8c159230 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c0c98d0 con 0x7fbb8c158e60 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb91fa9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0c9bb0 con 0x7fbb8c068540 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb8c159230 0x7fbb8c067e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54772/0 (socket says 172.21.15.7:54772) 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8affd700 1 -- 172.21.15.7:0/2715377138 learned_addr learned my addr 172.21.15.7:0/2715377138 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2715377138 >> v2:172.21.15.7:3300/0 conn(0x7fbb8c158e60 0x7fbb8c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8bfff700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c068540 0x7fbb8c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8a7fc700 1 -- 172.21.15.7:0/2715377138 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219971174 0 0) 0x7fbb8c0c95f0 con 0x7fbb8c159230 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8a7fc700 1 -- 172.21.15.7:0/2715377138 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb74003570 con 0x7fbb8c159230 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2715377138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c068540 msgr2=0x7fbb8c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c068540 0x7fbb8c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2715377138 >> v1:172.21.15.7:6789/0 conn(0x7fbb8c159230 legacy=0x7fbb8c067e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2715377138 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0c9e90 con 0x7fbb8c158e60 2023-10-06T11:22:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c158e60 0x7fbb8c0cb090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb800098e0 tx=0x7fbb800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a308a5268d08fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8a7fc700 1 -- 172.21.15.7:0/2715377138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80011630 con 0x7fbb8c158e60 2023-10-06T11:22:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8a7fc700 1 -- 172.21.15.7:0/2715377138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80011790 con 0x7fbb8c158e60 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.715+0000 7fbb8a7fc700 1 -- 172.21.15.7:0/2715377138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80011b30 con 0x7fbb8c158e60 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2715377138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c158e60 msgr2=0x7fbb8c0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c158e60 0x7fbb8c0cb090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb800098e0 tx=0x7fbb800091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2715377138 shutdown_connections 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c068540 0x7fbb8c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2715377138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c158e60 0x7fbb8c0cb090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2715377138 >> 172.21.15.7:0/2715377138 conn(0x7fbb8c06b8a0 msgr2=0x7fbb8c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2715377138 shutdown_connections 2023-10-06T11:22:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2715377138 wait complete. 2023-10-06T11:22:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 Processor -- start 2023-10-06T11:22:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- start start 2023-10-06T11:22:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb8c158e60 0x7fbb8c169260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 0x7fbb8c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c16d750 con 0x7fbb8c068540 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c1697a0 con 0x7fbb8c158e60 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb91fa9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0c49b0 con 0x7fbb8c159230 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb8b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb8b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41546/0 (socket says 172.21.15.7:41546) 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.719+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2517394879 learned_addr learned my addr 172.21.15.7:0/2517394879 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8bfff700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 0x7fbb8c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8affd700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb8c158e60 0x7fbb8c169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 msgr2=0x7fbb8c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 0x7fbb8c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb8c158e60 msgr2=0x7fbb8c169260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb8c158e60 0x7fbb8c169260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 -- 172.21.15.7:0/2517394879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80009420 con 0x7fbb8c068540 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8bfff700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 0x7fbb8c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb8b7fe700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c064a30 tx=0x7fbb80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80011b10 con 0x7fbb8c068540 2023-10-06T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800253f0 con 0x7fbb8c068540 2023-10-06T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800257b0 con 0x7fbb8c068540 2023-10-06T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0c3da0 con 0x7fbb8c068540 2023-10-06T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c0c4360 con 0x7fbb8c068540 2023-10-06T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.723+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c04be00 con 0x7fbb8c068540 2023-10-06T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.727+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb80025ba0 con 0x7fbb8c068540 2023-10-06T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.727+0000 7fbb737fe700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 0x7fbb78062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.727+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbb8007e520 con 0x7fbb8c068540 2023-10-06T11:22:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.727+0000 7fbb8affd700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 0x7fbb78062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.727+0000 7fbb8affd700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 0x7fbb78062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c15ca50 tx=0x7fbb84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.735+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb8004e6c0 con 0x7fbb8c068540 2023-10-06T11:22:17.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.935+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb8c0cbae0 con 0x7fbb8c068540 2023-10-06T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.943+0000 7fbb737fe700 1 -- 172.21.15.7:0/2517394879 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154130 (secure 0 0 0) 0x7fbb80051ce0 con 0x7fbb8c068540 2023-10-06T11:22:17.951 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 msgr2=0x7fbb78062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 0x7fbb78062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c15ca50 tx=0x7fbb84009040 comp rx=0 tx=0).stop 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 msgr2=0x7fbb8c16d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c064a30 tx=0x7fbb80017000 comp rx=0 tx=0).stop 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 shutdown_connections 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb78060550 0x7fbb78062a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb8c159230 0x7fbb8c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb8c158e60 0x7fbb8c169260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 --2- 172.21.15.7:0/2517394879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8c068540 0x7fbb8c16d210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 >> 172.21.15.7:0/2517394879 conn(0x7fbb8c06b8a0 msgr2=0x7fbb8c0c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 shutdown_connections 2023-10-06T11:22:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:17.947+0000 7fbb91fa9700 1 -- 172.21.15.7:0/2517394879 wait complete. 2023-10-06T11:22:17.966 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":33317},{"type":"v1","addr":"172.21.15.50:6833","nonce":33317}]},"active_addr":"172.21.15.50:6833/33317","active_change":"2023-10-06T11:21:43.791628+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3694782510}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":2634621881}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":330210624}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3120583769}]}]} 2023-10-06T11:22:17.969 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T11:22:17.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T11:22:17.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 Processor -- start 2023-10-06T11:22:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 -- start start 2023-10-06T11:22:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca00cbe20 0x7f0ca00cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 0x7f0ca00ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ca01567e0 0x7f0ca0156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca015b5a0 con 0x7f0ca01567e0 2023-10-06T11:22:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00caea0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca697c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca015a990 con 0x7f0ca00cbe20 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ca01567e0 0x7f0ca0156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ca01567e0 0x7f0ca0156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41558/0 (socket says 172.21.15.7:41558) 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 -- 172.21.15.7:0/2222846519 learned_addr learned my addr 172.21.15.7:0/2222846519 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/2222846519 >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 0x7f0ca00ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9ffff700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca00cbe20 0x7f0ca00cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 -- 172.21.15.7:0/2222846519 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca00cbe20 msgr2=0x7f0ca00cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca00cbe20 0x7f0ca00cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 -- 172.21.15.7:0/2222846519 >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 msgr2=0x7f0ca00ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 --2- 172.21.15.7:0/2222846519 >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 0x7f0ca00ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 -- 172.21.15.7:0/2222846519 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca015ac70 con 0x7f0ca01567e0 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/2222846519 >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 0x7f0ca00ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0ca4f19700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca01567e0 0x7f0ca0156c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00b19b0 tx=0x7f0c98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa10e2d24cf5c256 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9effd700 1 -- 172.21.15.7:0/2222846519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b860 con 0x7f0ca01567e0 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9effd700 1 -- 172.21.15.7:0/2222846519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800b9c0 con 0x7f0ca01567e0 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.067+0000 7f0c9effd700 1 -- 172.21.15.7:0/2222846519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800bd60 con 0x7f0ca01567e0 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- 172.21.15.7:0/2222846519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca01567e0 msgr2=0x7f0ca0156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca01567e0 0x7f0ca0156c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00b19b0 tx=0x7f0c98009250 comp rx=0 tx=0).stop 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- 172.21.15.7:0/2222846519 shutdown_connections 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca01567e0 0x7f0ca0156c60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- 172.21.15.7:0/2222846519 >> v2:172.21.15.50:3300/0 conn(0x7f0ca00c07c0 0x7f0ca00ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- 172.21.15.7:0/2222846519 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca00cbe20 0x7f0ca00cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- 172.21.15.7:0/2222846519 >> 172.21.15.7:0/2222846519 conn(0x7f0ca0066840 msgr2=0x7f0ca0066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- 172.21.15.7:0/2222846519 shutdown_connections 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- 172.21.15.7:0/2222846519 wait complete. 2023-10-06T11:22:18.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 Processor -- start 2023-10-06T11:22:18.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- start start 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 0x7f0ca0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca01567e0 0x7f0ca014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca014ca40 con 0x7f0ca00c07c0 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca014be30 con 0x7f0ca00cbe20 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca697c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca014c110 con 0x7f0ca01567e0 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41560/0 (socket says 172.21.15.7:41560) 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0c9ffff700 1 -- 172.21.15.7:0/3962157190 learned_addr learned my addr 172.21.15.7:0/3962157190 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0ca4f19700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca01567e0 0x7f0ca014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.071+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 0x7f0ca0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 -- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca01567e0 msgr2=0x7f0ca014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca01567e0 0x7f0ca014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 -- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 msgr2=0x7f0ca0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 0x7f0ca0158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 -- 172.21.15.7:0/3962157190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980094f0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 0x7f0ca0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c9ffff700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d890 tx=0x7f0c90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900125f0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90012750 con 0x7f0ca00c07c0 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca014c3f0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90012ad0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca014e370 con 0x7f0ca00c07c0 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0156590 con 0x7f0ca00c07c0 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.075+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0c9001e460 con 0x7f0ca00c07c0 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.079+0000 7f0c97fff700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 0x7f0c80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.079+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0c9007d2c0 con 0x7f0ca00c07c0 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.079+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 0x7f0c80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.079+0000 7f0c9f7fe700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 0x7f0c80062a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00c1f90 tx=0x7f0c880098f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.087+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0c9004d460 con 0x7f0ca00c07c0 2023-10-06T11:22:18.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.251+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ca004be00 con 0x7f0ca00c07c0 2023-10-06T11:22:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.255+0000 7f0c97fff700 1 -- 172.21.15.7:0/3962157190 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f0c90050a80 con 0x7f0ca00c07c0 2023-10-06T11:22:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:18.261 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 msgr2=0x7f0c80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 0x7f0c80062a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00c1f90 tx=0x7f0c880098f0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 msgr2=0x7f0ca01586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d890 tx=0x7f0c90008040 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 shutdown_connections 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0c80060550 0x7f0c80062a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0ca01567e0 0x7f0ca014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ca00cbe20 0x7f0ca0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 --2- 172.21.15.7:0/3962157190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ca00c07c0 0x7f0ca01586d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 >> 172.21.15.7:0/3962157190 conn(0x7f0ca0066840 msgr2=0x7f0ca014ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 shutdown_connections 2023-10-06T11:22:18.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.259+0000 7f0ca697c700 1 -- 172.21.15.7:0/3962157190 wait complete. 2023-10-06T11:22:18.277 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T11:22:18.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 Processor -- start 2023-10-06T11:22:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 -- start start 2023-10-06T11:22:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c1c156b60 0x7f8c1c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0cd4b0 con 0x7f8c1c0ab720 2023-10-06T11:22:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cd940 con 0x7f8c1c156b60 2023-10-06T11:22:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c208b9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0ac370 con 0x7f8c1c156f30 2023-10-06T11:22:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c1c156b60 0x7f8c1c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c19d9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c1c156b60 0x7f8c1c0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57674/0 (socket says 172.21.15.7:57674) 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 -- 172.21.15.7:0/2354910405 learned_addr learned my addr 172.21.15.7:0/2354910405 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1ad9d700 1 --1- 172.21.15.7:0/2354910405 >> v1:172.21.15.50:6789/0 conn(0x7f8c1c0ab720 0x7f8c1c0abba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:47350/0 (socket says 172.21.15.7:47350) 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 -- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 msgr2=0x7f8c1c0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 --2- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 -- 172.21.15.7:0/2354910405 >> v1:172.21.15.50:6789/0 conn(0x7f8c1c0ab720 legacy=0x7f8c1c0abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 -- 172.21.15.7:0/2354910405 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0ccb80 con 0x7f8c1c156b60 2023-10-06T11:22:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.375+0000 7f8c1a59c700 1 --2- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0c7990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009f20 tx=0x7f8c0c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c0d67b9f81dd1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c1959a700 1 -- 172.21.15.7:0/2354910405 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00e070 con 0x7f8c1c156b60 2023-10-06T11:22:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c1959a700 1 -- 172.21.15.7:0/2354910405 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00b9d0 con 0x7f8c1c156b60 2023-10-06T11:22:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c1959a700 1 -- 172.21.15.7:0/2354910405 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0092e0 con 0x7f8c1c156b60 2023-10-06T11:22:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 msgr2=0x7f8c1c0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0c7990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009f20 tx=0x7f8c0c009840 comp rx=0 tx=0).stop 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- 172.21.15.7:0/2354910405 shutdown_connections 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2354910405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0c7990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- 172.21.15.7:0/2354910405 >> 172.21.15.7:0/2354910405 conn(0x7f8c1c06b8a0 msgr2=0x7f8c1c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- 172.21.15.7:0/2354910405 shutdown_connections 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- 172.21.15.7:0/2354910405 wait complete. 2023-10-06T11:22:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 Processor -- start 2023-10-06T11:22:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.379+0000 7f8c208b9700 1 -- start start 2023-10-06T11:22:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cb5c0 con 0x7f8c1c0ab720 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cb8a0 con 0x7f8c1c156b60 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cbb80 con 0x7f8c1c156f30 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c1a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c1a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41576/0 (socket says 172.21.15.7:41576) 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c1a59c700 1 -- 172.21.15.7:0/2193061698 learned_addr learned my addr 172.21.15.7:0/2193061698 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c1ad9d700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 -- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 msgr2=0x7f8c1c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 -- 172.21.15.7:0/2193061698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 msgr2=0x7f8c1c14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 -- 172.21.15.7:0/2193061698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009ae0 con 0x7f8c1c156b60 2023-10-06T11:22:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c1a59c700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c19d9b700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0aae10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400e900 tx=0x7f8c040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04013680 con 0x7f8c1c156b60 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c040137e0 con 0x7f8c1c156b60 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0cbe60 con 0x7f8c1c156b60 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04013ba0 con 0x7f8c1c156b60 2023-10-06T11:22:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c0ab350 con 0x7f8c1c156b60 2023-10-06T11:22:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.383+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0ab570 con 0x7f8c1c156b60 2023-10-06T11:22:18.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.387+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8c04005a00 con 0x7f8c1c156b60 2023-10-06T11:22:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.387+0000 7f8c127fc700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 0x7f8c00062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.387+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8c0407d070 con 0x7f8c1c156b60 2023-10-06T11:22:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.387+0000 7f8c1a59c700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 0x7f8c00062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.387+0000 7f8c1a59c700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 0x7f8c00062a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009880 tx=0x7f8c0c018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.395+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8c0404d210 con 0x7f8c1c156b60 2023-10-06T11:22:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.559+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c1c0cb9e0 con 0x7f8c1c156b60 2023-10-06T11:22:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.563+0000 7f8c127fc700 1 -- 172.21.15.7:0/2193061698 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f8c1c0cb9e0 con 0x7f8c1c156b60 2023-10-06T11:22:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:18.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 msgr2=0x7f8c00062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 0x7f8c00062a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009880 tx=0x7f8c0c018000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 msgr2=0x7f8c1c0aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0aae10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400e900 tx=0x7f8c040091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 shutdown_connections 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8c00060580 0x7f8c00062a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8c1c156f30 0x7f8c1c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c156b60 0x7f8c1c0aae10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 --2- 172.21.15.7:0/2193061698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1c0ab720 0x7f8c1c14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 >> 172.21.15.7:0/2193061698 conn(0x7f8c1c06b8a0 msgr2=0x7f8c1c0c3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 shutdown_connections 2023-10-06T11:22:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.567+0000 7f8c208b9700 1 -- 172.21.15.7:0/2193061698 wait complete. 2023-10-06T11:22:18.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T11:22:18.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T11:22:18.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T11:22:18.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T11:22:18.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T11:22:18.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T11:22:18.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T11:22:18.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T11:22:18.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 Processor -- start 2023-10-06T11:22:18.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 -- start start 2023-10-06T11:22:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 Processor -- start 2023-10-06T11:22:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 -- start start 2023-10-06T11:22:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 0x7f65280c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6528157110 0x7f652806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65280c8c60 con 0x7f652815a890 2023-10-06T11:22:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65280c0a50 con 0x7f6528157110 2023-10-06T11:22:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652cc8e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652815ad20 con 0x7f6528156d40 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa444154500 0x7fa4440cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44415e3e0 con 0x7fa4440c7af0 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440c94b0 con 0x7fa4440cbab0 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa449a52700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4440accb0 con 0x7fa444154500 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f6525d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f652815a890 0x7f65280b0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54788/0 (socket says 172.21.15.7:54788) 2023-10-06T11:22:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f6525d9b700 1 -- 172.21.15.7:0/1181326376 learned_addr learned my addr 172.21.15.7:0/1181326376 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 Processor -- start 2023-10-06T11:22:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 -- start start 2023-10-06T11:22:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 0x7f985815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 --2- 172.21.15.7:0/1181326376 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 0x7f65280c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4437fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa4440c7af0 0x7fa44406dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54804/0 (socket says 172.21.15.7:54804) 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4437fe700 1 -- 172.21.15.7:0/3471962978 learned_addr learned my addr 172.21.15.7:0/3471962978 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4427fc700 1 --2- 172.21.15.7:0/3471962978 >> v2:172.21.15.7:3300/0 conn(0x7fa444154500 0x7fa4440cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98580cb820 0x7f98580cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98580cc1d0 con 0x7f98580696f0 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985806ed20 con 0x7f98580cb820 2023-10-06T11:22:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985f29f700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985815e800 con 0x7f98581577e0 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98580cb820 0x7f98580cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985d03b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 0x7f985815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98580cb820 0x7f98580cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41602/0 (socket says 172.21.15.7:41602) 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 -- 172.21.15.7:0/964219988 learned_addr learned my addr 172.21.15.7:0/964219988 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985d03b700 1 --2- 172.21.15.7:0/964219988 >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 0x7f985815e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57700/0 (socket says 172.21.15.7:57700) 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 -- 172.21.15.7:0/1181326376 >> v2:172.21.15.50:3300/0 conn(0x7f6528157110 msgr2=0x7f652806db70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 --2- 172.21.15.7:0/1181326376 >> v2:172.21.15.50:3300/0 conn(0x7f6528157110 0x7f652806db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 -- 172.21.15.7:0/1181326376 >> v1:172.21.15.7:6789/0 conn(0x7f652815a890 legacy=0x7f65280b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 -- 172.21.15.7:0/1181326376 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652815b000 con 0x7f6528156d40 2023-10-06T11:22:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 -- 172.21.15.7:0/964219988 >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 msgr2=0x7f985815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 --2- 172.21.15.7:0/964219988 >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 0x7f985815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 -- 172.21.15.7:0/964219988 >> v1:172.21.15.7:6789/0 conn(0x7f98580696f0 legacy=0x7f9858069b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f985c83a700 1 -- 172.21.15.7:0/964219988 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858161270 con 0x7f98580cb820 2023-10-06T11:22:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652659c700 1 --2- 172.21.15.7:0/1181326376 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 0x7f65280c8720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6520009e60 tx=0x7f6520009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321a063cccb7bb53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811544857 0 0) 0x7fa44415e3e0 con 0x7fa4440c7af0 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa42c003570 con 0x7fa4440c7af0 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222968300 0 0) 0x7fa4440c94b0 con 0x7fa4440cbab0 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4427fc700 1 -- 172.21.15.7:0/3471962978 >> v1:172.21.15.50:6789/0 conn(0x7fa4440cbab0 legacy=0x7fa444153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4427fc700 1 -- 172.21.15.7:0/3471962978 >> v1:172.21.15.7:6789/0 conn(0x7fa4440c7af0 legacy=0x7fa44406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7fa4427fc700 1 -- 172.21.15.7:0/3471962978 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44415dab0 con 0x7fa444154500 2023-10-06T11:22:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.727+0000 7f652559a700 1 -- 172.21.15.7:0/1181326376 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000e070 con 0x7f6528156d40 2023-10-06T11:22:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa4427fc700 1 --2- 172.21.15.7:0/3471962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa444154500 0x7fa4440cbe80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098e0 tx=0x7fa4340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c408a361961d7841 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652559a700 1 -- 172.21.15.7:0/1181326376 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000bc30 con 0x7f6528156d40 2023-10-06T11:22:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652559a700 1 -- 172.21.15.7:0/1181326376 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520012430 con 0x7f6528156d40 2023-10-06T11:22:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985c83a700 1 --2- 172.21.15.7:0/964219988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580cb820 0x7f98580cbc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f985806bbf0 tx=0x7f9854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44807b3a0c3399a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434011630 con 0x7fa444154500 2023-10-06T11:22:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434011790 con 0x7fa444154500 2023-10-06T11:22:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa441ffb700 1 -- 172.21.15.7:0/3471962978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434011b30 con 0x7fa444154500 2023-10-06T11:22:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f984ffff700 1 -- 172.21.15.7:0/964219988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854009e70 con 0x7f98580cb820 2023-10-06T11:22:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f984ffff700 1 -- 172.21.15.7:0/964219988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400b820 con 0x7f98580cb820 2023-10-06T11:22:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f984ffff700 1 -- 172.21.15.7:0/964219988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400bbc0 con 0x7f98580cb820 2023-10-06T11:22:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- 172.21.15.7:0/1181326376 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 msgr2=0x7f65280c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 --2- 172.21.15.7:0/1181326376 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 0x7f65280c8720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6520009e60 tx=0x7f6520009780 comp rx=0 tx=0).stop 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- 172.21.15.7:0/1181326376 shutdown_connections 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 --2- 172.21.15.7:0/1181326376 >> v2:172.21.15.50:3300/0 conn(0x7f6528157110 0x7f652806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 --2- 172.21.15.7:0/1181326376 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528156d40 0x7f65280c8720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- 172.21.15.7:0/1181326376 >> 172.21.15.7:0/1181326376 conn(0x7f6528066840 msgr2=0x7f6528066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 -- 172.21.15.7:0/3471962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa444154500 msgr2=0x7fa4440cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 --2- 172.21.15.7:0/3471962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa444154500 0x7fa4440cbe80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098e0 tx=0x7fa4340091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- 172.21.15.7:0/1181326376 shutdown_connections 2023-10-06T11:22:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 -- 172.21.15.7:0/3471962978 shutdown_connections 2023-10-06T11:22:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 --2- 172.21.15.7:0/3471962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa444154500 0x7fa4440cbe80 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098e0 tx=0x7fa4340091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 -- 172.21.15.7:0/3471962978 >> 172.21.15.7:0/3471962978 conn(0x7fa444066840 msgr2=0x7fa444066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- 172.21.15.7:0/1181326376 wait complete. 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 -- 172.21.15.7:0/964219988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580cb820 msgr2=0x7f98580cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 --2- 172.21.15.7:0/964219988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580cb820 0x7f98580cbc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f985806bbf0 tx=0x7f9854009250 comp rx=0 tx=0).stop 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 -- 172.21.15.7:0/964219988 shutdown_connections 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 --2- 172.21.15.7:0/964219988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580cb820 0x7f98580cbc90 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f985806bbf0 tx=0x7f9854009250 comp rx=0 tx=0).stop 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 --2- 172.21.15.7:0/964219988 >> v2:172.21.15.50:3300/0 conn(0x7f98581577e0 0x7f985815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f985f29f700 1 -- 172.21.15.7:0/964219988 >> 172.21.15.7:0/964219988 conn(0x7f9858066840 msgr2=0x7f9858066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 -- 172.21.15.7:0/3471962978 shutdown_connections 2023-10-06T11:22:18.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 Processor -- start 2023-10-06T11:22:18.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7fa449a52700 1 -- 172.21.15.7:0/3471962978 wait complete. 2023-10-06T11:22:18.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f56a0f4a700 1 Processor -- start 2023-10-06T11:22:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 -- start start 2023-10-06T11:22:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.731+0000 7f652cc8e700 1 -- start start 2023-10-06T11:22:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- 172.21.15.7:0/964219988 shutdown_connections 2023-10-06T11:22:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- 172.21.15.7:0/964219988 wait complete. 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 Processor -- start 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 0x7f652814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528157110 0x7f652816ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f652815a890 0x7f652816d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652816d8b0 con 0x7f6528156d40 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280af6b0 con 0x7f652815a890 2023-10-06T11:22:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65280af990 con 0x7f6528157110 2023-10-06T11:22:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- start start 2023-10-06T11:22:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f569c159060 0x7f569c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 0x7f569c15c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0abc60 con 0x7f569c158c90 2023-10-06T11:22:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c15d7b0 con 0x7f569c159060 2023-10-06T11:22:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f56a0f4a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0ad240 con 0x7f569c0ab790 2023-10-06T11:22:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 0x7f652814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 0x7f652814ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41604/0 (socket says 172.21.15.7:41604) 2023-10-06T11:22:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 -- 172.21.15.7:0/904818692 learned_addr learned my addr 172.21.15.7:0/904818692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 Processor -- start 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 -- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528157110 msgr2=0x7f652816ce30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6528157110 0x7f652816ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 -- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f652815a890 msgr2=0x7f652816d370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f652815a890 0x7f652816d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6520009a20 con 0x7f6528156d40 2023-10-06T11:22:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- start start 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 0x7fa4441432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4440cbab0 0x7fa44415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa44416dbc0 0x7fa444155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa444162970 con 0x7fa4440c7af0 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa444162080 con 0x7fa4440cbab0 2023-10-06T11:22:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa444162320 con 0x7fa44416dbc0 2023-10-06T11:22:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 0x7fa4441432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 0x7fa4441432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41612/0 (socket says 172.21.15.7:41612) 2023-10-06T11:22:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 -- 172.21.15.7:0/134094355 learned_addr learned my addr 172.21.15.7:0/134094355 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652659c700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 0x7f652814ece0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6520013000 tx=0x7f6520019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa4437fe700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa44416dbc0 0x7fa444155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569bfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 0x7f569c15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569bfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 0x7f569c15c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43358/0 (socket says 172.21.15.7:43358) 2023-10-06T11:22:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569bfff700 1 -- 172.21.15.7:0/3836826819 learned_addr learned my addr 172.21.15.7:0/3836826819 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f5693fff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f569c159060 0x7f569c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569b7fe700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f569c158c90 0x7f569c0ab080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:47368/0 (socket says 172.21.15.7:47368) 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa4427fc700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4440cbab0 0x7fa44415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 -- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa44416dbc0 msgr2=0x7fa444155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa44416dbc0 0x7fa444155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 -- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4440cbab0 msgr2=0x7fa44415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4440cbab0 0x7fa44415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434009420 con 0x7fa4440c7af0 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa4427fc700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4440cbab0 0x7fa44415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa442ffd700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 0x7fa4441432e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b5e0 tx=0x7fa438009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 0x7f98581461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98581577e0 0x7f985815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f985815cb10 0x7f985815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858159620 con 0x7f98580696f0 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858158a10 con 0x7f98581577e0 2023-10-06T11:22:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985f29f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9858158cf0 con 0x7f985815cb10 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569affd700 1 -- 172.21.15.7:0/3836826819 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848299742 0 0) 0x7f569c0abc60 con 0x7f569c158c90 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f569affd700 1 -- 172.21.15.7:0/3836826819 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5688003570 con 0x7f569c158c90 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f5693fff700 1 -- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 msgr2=0x7f569c15c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f5693fff700 1 --2- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 0x7f569c15c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f5693fff700 1 -- 172.21.15.7:0/3836826819 >> v1:172.21.15.50:6789/0 conn(0x7f569c158c90 legacy=0x7f569c0ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f5693fff700 1 -- 172.21.15.7:0/3836826819 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0abf40 con 0x7f569c159060 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f5693fff700 1 --2- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c159060 0x7f569c0acd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f568c009910 tx=0x7f568c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5cb0ccbb44182c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520012e60 con 0x7f6528156d40 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652001d630 con 0x7f6528156d40 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652001d9f0 con 0x7f6528156d40 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65280afc70 con 0x7f6528156d40 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f652cc8e700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65280b0230 con 0x7f6528156d40 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 0x7f98581461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 0x7f98581461f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41618/0 (socket says 172.21.15.7:41618) 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 -- 172.21.15.7:0/3990872605 learned_addr learned my addr 172.21.15.7:0/3990872605 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985c83a700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98581577e0 0x7f985815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 -- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f985815cb10 msgr2=0x7f985815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f985815cb10 0x7f985815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7f985d03b700 1 -- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98581577e0 msgr2=0x7f985815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f985d03b700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98581577e0 0x7f985815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f985d03b700 1 -- 172.21.15.7:0/3990872605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540094f0 con 0x7f98580696f0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f985d03b700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 0x7f98581461f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e50 tx=0x7f985000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000cc20 con 0x7f98580696f0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000f040 con 0x7f98580696f0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850014480 con 0x7f98580696f0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f985f29f700 1 -- 172.21.15.7:0/3990872605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9858158fd0 con 0x7f98580696f0 2023-10-06T11:22:18.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f985f29f700 1 -- 172.21.15.7:0/3990872605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9858158620 con 0x7f98580696f0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438011630 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4380117b0 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438011b70 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa444162600 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.735+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa444143820 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa44404be00 con 0x7fa4440c7af0 2023-10-06T11:22:18.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f569affd700 1 -- 172.21.15.7:0/3836826819 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c011630 con 0x7f569c159060 2023-10-06T11:22:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f569affd700 1 -- 172.21.15.7:0/3836826819 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c011790 con 0x7f569c159060 2023-10-06T11:22:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f569affd700 1 -- 172.21.15.7:0/3836826819 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c011b30 con 0x7f569c159060 2023-10-06T11:22:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa43801a400 con 0x7fa4440c7af0 2023-10-06T11:22:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6520024070 con 0x7f6528156d40 2023-10-06T11:22:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f652804be00 con 0x7f6528156d40 2023-10-06T11:22:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa430060620 0x7fa430062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 -- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c159060 msgr2=0x7f569c0acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c159060 0x7f569c0acd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f568c009910 tx=0x7f568c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6508060620 0x7f6508062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f652007f360 con 0x7f6528156d40 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa43807cb60 con 0x7fa4440c7af0 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fa43006a270 0x7fa43006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7f650806a2f0 0x7f650806c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f650806cdf0 con 0x7f650806a2f0 2023-10-06T11:22:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6520081970 con 0x7f6528156d40 2023-10-06T11:22:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f98500147f0 con 0x7f98580696f0 2023-10-06T11:22:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa43006cd70 con 0x7fa43006a270 2023-10-06T11:22:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa43804cd80 con 0x7fa4440c7af0 2023-10-06T11:22:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f6525d9b700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6508060620 0x7f6508062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9844060770 0x7f9844062c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f985007d920 con 0x7f98580696f0 2023-10-06T11:22:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f6526d9d700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7f650806a2f0 0x7f650806c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa4427fc700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa430060620 0x7fa430062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 -- 172.21.15.7:0/3836826819 shutdown_connections 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c0ab790 0x7f569c15c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/3836826819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c159060 0x7f569c0acd00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 -- 172.21.15.7:0/3836826819 >> 172.21.15.7:0/3836826819 conn(0x7f569c068900 msgr2=0x7f569c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7f56a0f4a700 1 -- 172.21.15.7:0/3836826819 shutdown_connections 2023-10-06T11:22:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.739+0000 7fa4437fe700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fa43006a270 0x7fa43006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 -- 172.21.15.7:0/3836826819 wait complete. 2023-10-06T11:22:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f6526d9d700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7f650806a2f0 0x7f650806c720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985c83a700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9844060770 0x7f9844062c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985f29f700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7f98380015f0 0x7f9838003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985f29f700 1 -- 172.21.15.7:0/3990872605 --> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9838006b90 con 0x7f98380015f0 2023-10-06T11:22:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7fa4437fe700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fa43006a270 0x7fa43006c6a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7fa4427fc700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa430060620 0x7fa430062ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4441548d0 tx=0x7fa434016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985c83a700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9844060770 0x7f9844062c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f985806bbf0 tx=0x7f9854016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f6525d9b700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6508060620 0x7f6508062ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65180058d0 tx=0x7f6518007ae0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== osd.1 v2:172.21.15.7:6816/33388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f650806cdf0 con 0x7f650806a2f0 2023-10-06T11:22:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985d83c700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7f98380015f0 0x7f9838003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 Processor -- start 2023-10-06T11:22:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 -- start start 2023-10-06T11:22:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f985d83c700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7f98380015f0 0x7f9838003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f569c0ab790 0x7f569c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c158c90 0x7f569c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c159060 0x7f569c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b0420 con 0x7f569c0ab790 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0cbfa0 con 0x7f569c158c90 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f56a0f4a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c15be20 con 0x7f569c159060 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f569c0ab790 0x7f569c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== osd.5 v2:172.21.15.50:6816/33666 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa43006cd70 con 0x7fa43006a270 2023-10-06T11:22:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f569c0ab790 0x7f569c0cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2023-10-06T11:22:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f569b7fe700 1 -- 172.21.15.7:0/1095696568 learned_addr learned my addr 172.21.15.7:0/1095696568 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f569bfff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c159060 0x7f569c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7fdc47fff700 1 Processor -- start 2023-10-06T11:22:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f1418966700 1 Processor -- start 2023-10-06T11:22:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7fdc47fff700 1 -- start start 2023-10-06T11:22:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f1418966700 1 -- start start 2023-10-06T11:22:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.743+0000 7f5693fff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c158c90 0x7f569c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 -- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c159060 msgr2=0x7f569c153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f569c159060 0x7f569c153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 -- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f569c0ab790 msgr2=0x7f569c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f569c0ab790 0x7f569c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c009420 con 0x7f569c158c90 2023-10-06T11:22:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 Processor -- start 2023-10-06T11:22:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 -- start start 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f5693fff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c158c90 0x7f569c15a040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f568c016000 tx=0x7f568c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4815d410 0x7fdc48160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc47fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480c9410 con 0x7fdc4815d040 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc47fff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc48161830 con 0x7fdc481612b0 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc47fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48161b10 con 0x7fdc4815d410 2023-10-06T11:22:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c01f070 con 0x7f569c158c90 2023-10-06T11:22:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdc4815d040 0x7fdc480c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54826/0 (socket says 172.21.15.7:54826) 2023-10-06T11:22:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 -- 172.21.15.7:0/360847111 learned_addr learned my addr 172.21.15.7:0/360847111 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1414154bc0 0x7f1414154f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f14141554d0 0x7f14140af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c011900 con 0x7f569c158c90 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c023410 con 0x7f569c158c90 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14140afcc0 0x7f14141589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14140abba0 con 0x7f1414154bc0 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14140b01d0 con 0x7f14141554d0 2023-10-06T11:22:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1418966700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140ab270 con 0x7f14140afcc0 2023-10-06T11:22:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498998271 0 0) 0x7fdc480c9410 con 0x7fdc4815d040 2023-10-06T11:22:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc30003570 con 0x7fdc4815d040 2023-10-06T11:22:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443667709 0 0) 0x7fdc48161830 con 0x7fdc481612b0 2023-10-06T11:22:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc480c9410 con 0x7fdc481612b0 2023-10-06T11:22:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926580435 0 0) 0x7fdc30003570 con 0x7fdc4815d040 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4815d410 msgr2=0x7fdc48160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 0x7f7f900aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f90156a50 0x7f7f900a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900aa620 con 0x7f7f9015c9a0 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f900a97a0 con 0x7f7f90156a50 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96a97700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f900a9a10 con 0x7f7f90156680 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96296700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f9015c9a0 0x7f7f901593a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54838/0 (socket says 172.21.15.7:54838) 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f96296700 1 -- 172.21.15.7:0/3199116138 learned_addr learned my addr 172.21.15.7:0/3199116138 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3199116138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 0x7f7f900aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95294700 1 --2- 172.21.15.7:0/3199116138 >> v2:172.21.15.7:3300/0 conn(0x7f7f90156a50 0x7f7f900a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1414154bc0 0x7f1414154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1414154bc0 0x7f1414154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41636/0 (socket says 172.21.15.7:41636) 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 -- 172.21.15.7:0/443959421 learned_addr learned my addr 172.21.15.7:0/443959421 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f141259c700 1 --2- 172.21.15.7:0/443959421 >> v2:172.21.15.50:3300/0 conn(0x7f14141554d0 0x7f14140af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 -- 172.21.15.7:0/443959421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14140afcc0 msgr2=0x7f14141589e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 --2- 172.21.15.7:0/443959421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14140afcc0 0x7f14141589e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 -- 172.21.15.7:0/443959421 >> v2:172.21.15.50:3300/0 conn(0x7f14141554d0 msgr2=0x7f14140af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 --2- 172.21.15.7:0/443959421 >> v2:172.21.15.50:3300/0 conn(0x7f14141554d0 0x7f14140af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 -- 172.21.15.7:0/443959421 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140ab550 con 0x7f1414154bc0 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc467fc700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4815d410 0x7fdc48160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4815d410 0x7fdc48160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 >> v1:172.21.15.50:6789/0 conn(0x7fdc481612b0 legacy=0x7fdc48067e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480c9660 con 0x7fdc4815d040 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7fdc400017f0 con 0x7fdc4815d040 2023-10-06T11:22:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 >> v1:172.21.15.7:6789/0 conn(0x7fdc4815d040 legacy=0x7fdc480c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc30003c70 0x7fdc30004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc30004810 0x7fdc30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc300053c0 0x7fdc3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48161b10 con 0x7fdc300053c0 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000c2f0 con 0x7fdc30004810 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000c4b0 con 0x7fdc30003c70 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc467fc700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4815d410 0x7fdc48160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f94a93700 1 -- 172.21.15.7:0/3199116138 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378846048 0 0) 0x7f7f900aa620 con 0x7f7f9015c9a0 2023-10-06T11:22:18.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f94a93700 1 -- 172.21.15.7:0/3199116138 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c003570 con 0x7f7f9015c9a0 2023-10-06T11:22:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1412d9d700 1 --2- 172.21.15.7:0/443959421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1414154bc0 0x7f1414154f90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14140c82c0 tx=0x7f1404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4fb70645b44c8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 -- 172.21.15.7:0/3199116138 >> v2:172.21.15.7:3300/0 conn(0x7f7f90156a50 msgr2=0x7f7f900a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3199116138 >> v2:172.21.15.7:3300/0 conn(0x7f7f90156a50 0x7f7f900a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 -- 172.21.15.7:0/3199116138 >> v1:172.21.15.7:6789/0 conn(0x7f7f9015c9a0 legacy=0x7f7f901593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 -- 172.21.15.7:0/3199116138 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900a9cf0 con 0x7f7f90156680 2023-10-06T11:22:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1411d9b700 1 -- 172.21.15.7:0/443959421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400b830 con 0x7f1414154bc0 2023-10-06T11:22:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1411d9b700 1 -- 172.21.15.7:0/443959421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400b990 con 0x7f1414154bc0 2023-10-06T11:22:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc467fc700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc30004810 0x7fdc30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc30003c70 0x7fdc30004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f1411d9b700 1 -- 172.21.15.7:0/443959421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400bd50 con 0x7f1414154bc0 2023-10-06T11:22:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f8cfe773700 1 Processor -- start 2023-10-06T11:22:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f8cfe773700 1 -- start start 2023-10-06T11:22:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3199116138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 0x7f7f900aab60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0098e0 tx=0x7f7f8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c7b43ff4f94bfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c15c100 con 0x7f569c158c90 2023-10-06T11:22:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== osd.4 v2:172.21.15.50:6800/33665 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9838006b90 con 0x7f98380015f0 2023-10-06T11:22:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 -- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc30004810 msgr2=0x7fdc30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc30004810 0x7fdc30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 -- 172.21.15.7:0/360847111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc300053c0 msgr2=0x7fdc3000ba60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:22:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc300053c0 0x7fdc3000ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 -- 172.21.15.7:0/360847111 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30003570 con 0x7fdc30003c70 2023-10-06T11:22:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c15ba40 con 0x7f569c158c90 2023-10-06T11:22:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc30003c70 0x7fdc30004060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000a820 tx=0x7fdc400020c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6807104aaf6287fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- 172.21.15.7:0/443959421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1414154bc0 msgr2=0x7f1414154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- 172.21.15.7:0/443959421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1414154bc0 0x7f1414154f90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14140c82c0 tx=0x7f1404009250 comp rx=0 tx=0).stop 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- 172.21.15.7:0/443959421 shutdown_connections 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- 172.21.15.7:0/443959421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14140afcc0 0x7f14141589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- 172.21.15.7:0/443959421 >> v2:172.21.15.50:3300/0 conn(0x7f14141554d0 0x7f14140af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- 172.21.15.7:0/443959421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1414154bc0 0x7f1414154f90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- 172.21.15.7:0/443959421 >> 172.21.15.7:0/443959421 conn(0x7f1414068900 msgr2=0x7f14140cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc400070f0 con 0x7fdc30003c70 2023-10-06T11:22:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc40007250 con 0x7fdc30003c70 2023-10-06T11:22:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f7f94a93700 1 -- 172.21.15.7:0/3199116138 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c011630 con 0x7f7f90156680 2023-10-06T11:22:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f7f94a93700 1 -- 172.21.15.7:0/3199116138 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c011790 con 0x7f7f90156680 2023-10-06T11:22:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- 172.21.15.7:0/443959421 shutdown_connections 2023-10-06T11:22:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc45ffb700 1 -- 172.21.15.7:0/360847111 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40007610 con 0x7fdc30003c70 2023-10-06T11:22:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f7f94a93700 1 -- 172.21.15.7:0/3199116138 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c011b30 con 0x7f7f90156680 2023-10-06T11:22:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f569c15bc30 con 0x7f569c158c90 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc30003c70 msgr2=0x7fdc30004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 --2- 172.21.15.7:0/360847111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc30003c70 0x7fdc30004060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000a820 tx=0x7fdc400020c0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc477fe700 1 -- 172.21.15.7:0/360847111 reap_dead start 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- 172.21.15.7:0/360847111 shutdown_connections 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- 172.21.15.7:0/360847111 >> 172.21.15.7:0/360847111 conn(0x7fdc4806f380 msgr2=0x7fdc4806b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- 172.21.15.7:0/360847111 shutdown_connections 2023-10-06T11:22:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- 172.21.15.7:0/443959421 wait complete. 2023-10-06T11:22:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- 172.21.15.7:0/360847111 wait complete. 2023-10-06T11:22:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 Processor -- start 2023-10-06T11:22:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- start start 2023-10-06T11:22:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f7f96a97700 1 -- 172.21.15.7:0/3199116138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 msgr2=0x7f7f900aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3199116138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 0x7f7f900aab60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0098e0 tx=0x7f7f8c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f568c0193b0 con 0x7f569c158c90 2023-10-06T11:22:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f567c060550 0x7f567c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f568c07e1e0 con 0x7f569c158c90 2023-10-06T11:22:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 Processor -- start 2023-10-06T11:22:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14140afcc0 0x7f1414148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1414154bc0 0x7f14140c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 0x7f14140c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414169c20 con 0x7f14140afcc0 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140c8ea0 con 0x7f1414154bc0 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f1418966700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140c9180 con 0x7f14141554d0 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f141359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 0x7f14140c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f141359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 0x7f14140c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43424/0 (socket says 172.21.15.7:43424) 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f141359e700 1 -- 172.21.15.7:0/2221715420 learned_addr learned my addr 172.21.15.7:0/2221715420 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f141259c700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1414154bc0 0x7f14140c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f567c06a220 0x7f567c06c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f567c06cd20 con 0x7f567c06a220 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568c07e5b0 con 0x7f569c158c90 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f569bfff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f567c06a220 0x7f567c06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f569b7fe700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f567c060550 0x7f567c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f569bfff700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f567c06a220 0x7f567c06c650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f569b7fe700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f567c060550 0x7f567c062a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5684007fd0 tx=0x7f5684009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7fdc47fff700 1 -- start start 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4814ab50 0x7fdc480c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4814af20 0x7fdc480c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4815e920 0x7fdc480c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4815e490 con 0x7fdc4815e920 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480c68e0 con 0x7fdc4814ab50 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc467fc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4814af20 0x7fdc480c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4815e920 0x7fdc480c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc467fc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4814af20 0x7fdc480c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc467fc700 1 -- 172.21.15.7:0/4277416798 learned_addr learned my addr 172.21.15.7:0/4277416798 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4814ab50 0x7fdc480c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480c3cf0 con 0x7fdc4814af20 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 -- 172.21.15.7:0/3199116138 shutdown_connections 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3199116138 >> v2:172.21.15.7:3300/0 conn(0x7f7f90156a50 0x7f7f900a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3199116138 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156680 0x7f7f900aab60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 -- 172.21.15.7:0/3199116138 >> 172.21.15.7:0/3199116138 conn(0x7f7f90068900 msgr2=0x7f7f900cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 -- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1414154bc0 msgr2=0x7f14140c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1414154bc0 0x7f14140c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 -- 172.21.15.7:0/2221715420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14140afcc0 msgr2=0x7f1414148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14140afcc0 0x7f1414148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040094f0 con 0x7f14141554d0 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f141359e700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 0x7f14140c8870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f140c00bb60 tx=0x7f140c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.747+0000 7f8cfe773700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8cf80ce920 0x7f8cf80ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cfe773700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cfe773700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf8069f50 con 0x7f8cf80ce550 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cfe773700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf806a230 con 0x7f8cf80ce920 2023-10-06T11:22:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cfe773700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf806a510 con 0x7f8cf806afd0 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8cf80ce550 0x7f8cf80cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54854/0 (socket says 172.21.15.7:54854) 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf7fff700 1 -- 172.21.15.7:0/1353378883 learned_addr learned my addr 172.21.15.7:0/1353378883 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1353378883 >> v2:172.21.15.50:3300/0 conn(0x7f8cf80ce920 0x7f8cf80ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 msgr2=0x7f8cf80ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1353378883 >> v1:172.21.15.7:6789/0 conn(0x7f8cf80ce550 legacy=0x7f8cf80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1353378883 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf806a7f0 con 0x7f8cf80ce920 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf80ccfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0014b10 tx=0x7f8cf00143c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dff94b2d4f9eb59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1353378883 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000fd60 con 0x7f8cf80ce920 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== osd.0 v2:172.21.15.7:6808/33385 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f567c06cd20 con 0x7f567c06a220 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1353378883 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf00138a0 con 0x7f8cf80ce920 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.751+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1353378883 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0013c40 con 0x7f8cf80ce920 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 -- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 msgr2=0x7f8cf80ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 --2- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf80ccfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0014b10 tx=0x7f8cf00143c0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 -- 172.21.15.7:0/1353378883 shutdown_connections 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 --2- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 --2- 172.21.15.7:0/1353378883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf80ccfc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 -- 172.21.15.7:0/1353378883 >> 172.21.15.7:0/1353378883 conn(0x7f8cf8067e30 msgr2=0x7f8cf8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00c8c0 con 0x7f14141554d0 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14140c9460 con 0x7f14141554d0 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c00ca60 con 0x7f14141554d0 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00ce20 con 0x7f14141554d0 2023-10-06T11:22:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14140c9a20 con 0x7f14141554d0 2023-10-06T11:22:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 -- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4814af20 msgr2=0x7fdc480c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdc4814af20 0x7fdc480c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 -- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4814ab50 msgr2=0x7fdc480c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4814ab50 0x7fdc480c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4815d880 con 0x7fdc4815e920 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc477fe700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4815e920 0x7fdc480c8bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc48161680 tx=0x7fdc38007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800cea0 con 0x7fdc4815e920 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4815db60 con 0x7fdc4815e920 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4815e120 con 0x7fdc4815e920 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13f4000fd0 con 0x7f14141554d0 2023-10-06T11:22:18.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc38007280 con 0x7fdc4815e920 2023-10-06T11:22:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38007620 con 0x7fdc4815e920 2023-10-06T11:22:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc28000fd0 con 0x7fdc4815e920 2023-10-06T11:22:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 -- 172.21.15.7:0/3199116138 shutdown_connections 2023-10-06T11:22:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f8cfe773700 1 -- 172.21.15.7:0/1353378883 shutdown_connections 2023-10-06T11:22:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.755+0000 7f7f96a97700 1 -- 172.21.15.7:0/3199116138 wait complete. 2023-10-06T11:22:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdc38005a20 con 0x7fdc4815e920 2023-10-06T11:22:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f140c035a70 con 0x7f14141554d0 2023-10-06T11:22:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f7f96a97700 1 Processor -- start 2023-10-06T11:22:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdc34060580 0x7fdc34062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f8cfe773700 1 -- 172.21.15.7:0/1353378883 wait complete. 2023-10-06T11:22:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f14000605d0 0x7f1400062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f140c07cf90 con 0x7f14141554d0 2023-10-06T11:22:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdc3807c630 con 0x7fdc4815e920 2023-10-06T11:22:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f1412d9d700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f14000605d0 0x7f1400062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7f140006a2a0 0x7f140006c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f140006cda0 con 0x7f140006a2a0 2023-10-06T11:22:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140c07d2c0 con 0x7f14141554d0 2023-10-06T11:22:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdc34060580 0x7fdc34062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7fdc3406a250 0x7fdc3406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc3406cd50 con 0x7fdc3406a250 2023-10-06T11:22:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3807ca00 con 0x7fdc4815e920 2023-10-06T11:22:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f141259c700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7f140006a2a0 0x7f140006c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc467fc700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7fdc3406a250 0x7fdc3406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7fdc46ffd700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdc34060580 0x7fdc34062a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc4804c480 tx=0x7fdc40016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f1412d9d700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f14000605d0 0x7f1400062a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14140c82c0 tx=0x7f1404016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f8cfe773700 1 Processor -- start 2023-10-06T11:22:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f141259c700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7f140006a2a0 0x7f140006c6d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f8cfe773700 1 -- start start 2023-10-06T11:22:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf80ce550 0x7f8cf814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf8151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80c9040 con 0x7f8cf80ce550 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf815adb0 con 0x7f8cf80ce920 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfe773700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8159ef0 con 0x7f8cf806afd0 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.759+0000 7f7f96a97700 1 -- start start 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f90156680 0x7f7f901542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156a50 0x7f7f90178f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 0x7f7f90153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901522f0 con 0x7f7f9015c9a0 2023-10-06T11:22:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f901794c0 con 0x7f7f90156680 2023-10-06T11:22:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9014fa30 con 0x7f7f90156a50 2023-10-06T11:22:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== osd.7 v2:172.21.15.50:6817/33667 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f140006cda0 con 0x7f140006a2a0 2023-10-06T11:22:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7fdc467fc700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7fdc3406a250 0x7fdc3406c680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 0x7f7f90153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 0x7f7f90153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41666/0 (socket says 172.21.15.7:41666) 2023-10-06T11:22:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 -- 172.21.15.7:0/3563080073 learned_addr learned my addr 172.21.15.7:0/3563080073 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f90156680 0x7f7f901542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf8151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf8151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57794/0 (socket says 172.21.15.7:57794) 2023-10-06T11:22:18.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 -- 172.21.15.7:0/778880871 learned_addr learned my addr 172.21.15.7:0/778880871 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:18.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f95294700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156a50 0x7f7f90178f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf80ce550 0x7f8cf814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 -- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 msgr2=0x7f8cf8150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf8150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 -- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf80ce550 msgr2=0x7f8cf814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf80ce550 0x7f8cf814b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 -- 172.21.15.7:0/778880871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0014600 con 0x7f8cf80ce920 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8cf806afd0 0x7f8cf8150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== osd.2 v2:172.21.15.7:6824/33406 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdc3406cd50 con 0x7fdc3406a250 2023-10-06T11:22:18.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f8cfcd10700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf8151080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b590 tx=0x7f8ce8006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 -- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156a50 msgr2=0x7f7f90178f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7f90156a50 0x7f7f90178f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 -- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f90156680 msgr2=0x7f7f901542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f90156680 0x7f7f901542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 -- 172.21.15.7:0/3563080073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009420 con 0x7f7f9015c9a0 2023-10-06T11:22:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96296700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 0x7f7f90153440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800cbc0 tx=0x7f7f880019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800a7a0 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9014fd10 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901502d0 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8800a900 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.763+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800aca0 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.767+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7f88007300 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.767+0000 7f7f85ffb700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7f74060550 0x7f7f74062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.767+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7f8807cf70 con 0x7f7f9015c9a0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.767+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7f74060550 0x7f7f74062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.767+0000 7f7f95a95700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7f74060550 0x7f7f74062a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0093f0 tx=0x7f7f8c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800f5e0 con 0x7f8cf80ce920 2023-10-06T11:22:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce800f740 con 0x7f8cf80ce920 2023-10-06T11:22:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf815a1a0 con 0x7f8cf80ce920 2023-10-06T11:22:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800fb00 con 0x7f8cf80ce920 2023-10-06T11:22:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf815a760 con 0x7f8cf80ce920 2023-10-06T11:22:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 --> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f652804be00 con 0x7f650806a2f0 2023-10-06T11:22:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f65167fc700 1 -- 172.21.15.7:0/904818692 <== osd.1 v2:172.21.15.7:6816/33388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f652804be00 con 0x7f650806a2f0 2023-10-06T11:22:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 --> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa444162790 con 0x7fa43006a270 2023-10-06T11:22:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f7f700015f0 0x7f7f70003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 --> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f70006b90 con 0x7f7f700015f0 2023-10-06T11:22:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f7f95294700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f7f700015f0 0x7f7f70003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7fa42affd700 1 -- 172.21.15.7:0/134094355 <== osd.5 v2:172.21.15.50:6816/33666 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa444162790 con 0x7fa43006a270 2023-10-06T11:22:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.771+0000 7f7f95294700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f7f700015f0 0x7f7f70003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fa43006a270 msgr2=0x7fa43006c6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fa43006a270 0x7fa43006c6a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa430060620 msgr2=0x7fa430062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa430060620 0x7fa430062ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4441548d0 tx=0x7fa434016000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 msgr2=0x7fa4441432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 --2- 172.21.15.7:0/134094355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4440c7af0 0x7fa4441432e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b5e0 tx=0x7fa438009120 comp rx=0 tx=0).stop 2023-10-06T11:22:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa4437fe700 1 -- 172.21.15.7:0/134094355 reap_dead start 2023-10-06T11:22:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8ce801a400 con 0x7f8cf80ce920 2023-10-06T11:22:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 shutdown_connections 2023-10-06T11:22:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 >> 172.21.15.7:0/134094355 conn(0x7fa444066840 msgr2=0x7fa4440b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 shutdown_connections 2023-10-06T11:22:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7fa449a52700 1 -- 172.21.15.7:0/134094355 wait complete. 2023-10-06T11:22:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f8cdffff700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8ce4060620 0x7f8ce4062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8ce807c6e0 con 0x7f8cf80ce920 2023-10-06T11:22:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8ce4060620 0x7f8ce4062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== osd.6 v2:172.21.15.50:6808/33664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7f70006b90 con 0x7f7f700015f0 2023-10-06T11:22:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8ce4060620 0x7f8ce4062ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007fd0 tx=0x7f8cec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.775+0000 7f985f29f700 1 -- 172.21.15.7:0/3990872605 --> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9838005c80 con 0x7f98380015f0 2023-10-06T11:22:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f8cfe773700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f8cf80cc7a0 0x7f8cf80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 --> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cf8069740 con 0x7f8cf80cc7a0 2023-10-06T11:22:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f8cf80cc7a0 0x7f8cf80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f984d7fa700 1 -- 172.21.15.7:0/3990872605 <== osd.4 v2:172.21.15.50:6800/33665 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9838005c80 con 0x7f98380015f0 2023-10-06T11:22:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f8cf80cc7a0 0x7f8cf80c7100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== osd.3 v2:172.21.15.7:6800/33384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8cf8069740 con 0x7f8cf80cc7a0 2023-10-06T11:22:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7f98380015f0 msgr2=0x7f9838003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7f98380015f0 0x7f9838003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9844060770 msgr2=0x7f9844062c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9844060770 0x7f9844062c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f985806bbf0 tx=0x7f9854016000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 msgr2=0x7f98581461f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 --2- 172.21.15.7:0/3990872605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98580696f0 0x7f98581461f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e50 tx=0x7f985000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f985d83c700 1 -- 172.21.15.7:0/3990872605 reap_dead start 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 shutdown_connections 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 >> 172.21.15.7:0/3990872605 conn(0x7f9858066840 msgr2=0x7f98580ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 shutdown_connections 2023-10-06T11:22:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f9842ffd700 1 -- 172.21.15.7:0/3990872605 wait complete. 2023-10-06T11:22:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7f650806a2f0 msgr2=0x7f650806c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.779+0000 7f6507fff700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7f650806a2f0 0x7f650806c720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6508060620 msgr2=0x7f6508062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f6507fff700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6508060620 0x7f6508062ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65180058d0 tx=0x7f6518007ae0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 --> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f569c06c710 con 0x7f567c06a220 2023-10-06T11:22:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 msgr2=0x7f652814ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f6507fff700 1 --2- 172.21.15.7:0/904818692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528156d40 0x7f652814ece0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6520013000 tx=0x7f6520019000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f6526d9d700 1 -- 172.21.15.7:0/904818692 reap_dead start 2023-10-06T11:22:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56937fe700 1 -- 172.21.15.7:0/1095696568 <== osd.0 v2:172.21.15.7:6808/33385 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f569c06c710 con 0x7f567c06a220 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f567c06a220 msgr2=0x7f567c06c650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f567c06a220 0x7f567c06c650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f567c060550 msgr2=0x7f567c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f567c060550 0x7f567c062a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5684007fd0 tx=0x7f5684009040 comp rx=0 tx=0).stop 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c158c90 msgr2=0x7f569c15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 --2- 172.21.15.7:0/1095696568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f569c158c90 0x7f569c15a040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f568c016000 tx=0x7f568c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f569bfff700 1 -- 172.21.15.7:0/1095696568 reap_dead start 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 shutdown_connections 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 >> 172.21.15.7:0/1095696568 conn(0x7f569c068900 msgr2=0x7f569c156340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 shutdown_connections 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.783+0000 7f56a0f4a700 1 -- 172.21.15.7:0/1095696568 wait complete. 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.787+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 --> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc28002d30 con 0x7fdc3406a250 2023-10-06T11:22:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.787+0000 7fdc2effd700 1 -- 172.21.15.7:0/4277416798 <== osd.2 v2:172.21.15.7:6824/33406 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdc28002d30 con 0x7fdc3406a250 2023-10-06T11:22:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 shutdown_connections 2023-10-06T11:22:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 >> 172.21.15.7:0/904818692 conn(0x7f6528066840 msgr2=0x7f6528157570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7fdc3406a250 msgr2=0x7fdc3406c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7fdc3406a250 0x7fdc3406c680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.834 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:18.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:22:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdc34060580 msgr2=0x7fdc34062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdc34060580 0x7fdc34062a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc4804c480 tx=0x7fdc40016000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4815e920 msgr2=0x7fdc480c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4277416798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4815e920 0x7fdc480c8bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc48161680 tx=0x7fdc38007d10 comp rx=0 tx=0).stop 2023-10-06T11:22:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc477fe700 1 -- 172.21.15.7:0/4277416798 reap_dead start 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 shutdown_connections 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 shutdown_connections 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 >> 172.21.15.7:0/4277416798 conn(0x7fdc4806f380 msgr2=0x7fdc48159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 shutdown_connections 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7fdc47fff700 1 -- 172.21.15.7:0/4277416798 wait complete. 2023-10-06T11:22:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f6507fff700 1 -- 172.21.15.7:0/904818692 wait complete. 2023-10-06T11:22:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 --> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13f4002d30 con 0x7f140006a2a0 2023-10-06T11:22:18.837 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2023-10-06T11:22:18.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:22:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f13faffd700 1 -- 172.21.15.7:0/2221715420 <== osd.7 v2:172.21.15.50:6817/33667 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f13f4002d30 con 0x7f140006a2a0 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7f140006a2a0 msgr2=0x7f140006c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7f140006a2a0 0x7f140006c6d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f14000605d0 msgr2=0x7f1400062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f14000605d0 0x7f1400062a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14140c82c0 tx=0x7f1404016000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 msgr2=0x7f14140c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 --2- 172.21.15.7:0/2221715420 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f14141554d0 0x7f14140c8870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f140c00bb60 tx=0x7f140c0097f0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f141359e700 1 -- 172.21.15.7:0/2221715420 reap_dead start 2023-10-06T11:22:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 shutdown_connections 2023-10-06T11:22:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.791+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 >> 172.21.15.7:0/2221715420 conn(0x7f1414068900 msgr2=0x7f141406d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.795+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 shutdown_connections 2023-10-06T11:22:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.795+0000 7f1418966700 1 -- 172.21.15.7:0/2221715420 wait complete. 2023-10-06T11:22:18.840 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:18.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:22:18.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.799+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 --> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f70005c80 con 0x7f7f700015f0 2023-10-06T11:22:18.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f85ffb700 1 -- 172.21.15.7:0/3563080073 <== osd.6 v2:172.21.15.50:6808/33664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f70005c80 con 0x7f7f700015f0 2023-10-06T11:22:18.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f7f700015f0 msgr2=0x7f7f70003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f7f700015f0 0x7f7f70003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7f74060550 msgr2=0x7f7f74062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7f74060550 0x7f7f74062a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0093f0 tx=0x7f7f8c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 msgr2=0x7f7f90153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 --2- 172.21.15.7:0/3563080073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9015c9a0 0x7f7f90153440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800cbc0 tx=0x7f7f880019c0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96296700 1 -- 172.21.15.7:0/3563080073 reap_dead start 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 shutdown_connections 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 >> 172.21.15.7:0/3563080073 conn(0x7f7f90068900 msgr2=0x7f7f90159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 shutdown_connections 2023-10-06T11:22:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.803+0000 7f7f96a97700 1 -- 172.21.15.7:0/3563080073 wait complete. 2023-10-06T11:22:18.844 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:18.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:22:18.845 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:18.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:22:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.807+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 --> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cf814f7b0 con 0x7f8cf80cc7a0 2023-10-06T11:22:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cdffff700 1 -- 172.21.15.7:0/778880871 <== osd.3 v2:172.21.15.7:6800/33384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8cf814f7b0 con 0x7f8cf80cc7a0 2023-10-06T11:22:18.847 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:18.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f8cf80cc7a0 msgr2=0x7f8cf80c7100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f8cf80cc7a0 0x7f8cf80c7100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8ce4060620 msgr2=0x7f8ce4062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8ce4060620 0x7f8ce4062ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007fd0 tx=0x7f8cec009040 comp rx=0 tx=0).stop 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 msgr2=0x7f8cf8151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 --2- 172.21.15.7:0/778880871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8cf80ce920 0x7f8cf8151080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b590 tx=0x7f8ce8006f90 comp rx=0 tx=0).stop 2023-10-06T11:22:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfcd10700 1 -- 172.21.15.7:0/778880871 reap_dead start 2023-10-06T11:22:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 shutdown_connections 2023-10-06T11:22:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 >> 172.21.15.7:0/778880871 conn(0x7f8cf8067e30 msgr2=0x7f8cf80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 shutdown_connections 2023-10-06T11:22:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:18.811+0000 7f8cfe773700 1 -- 172.21.15.7:0/778880871 wait complete. 2023-10-06T11:22:18.849 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:18.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:22:18.851 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2023-10-06T11:22:18.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:22:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 Processor -- start 2023-10-06T11:22:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 -- start start 2023-10-06T11:22:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd8300afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300abba0 con 0x7fd830154bc0 2023-10-06T11:22:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300b01d0 con 0x7fd8301589e0 2023-10-06T11:22:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd836ed6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300ab270 con 0x7fd8300af780 2023-10-06T11:22:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd835ed4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd830154bc0 0x7fd830154f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54868/0 (socket says 172.21.15.7:54868) 2023-10-06T11:22:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd835ed4700 1 -- 172.21.15.7:0/953988408 learned_addr learned my addr 172.21.15.7:0/953988408 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd8356d3700 1 --2- 172.21.15.7:0/953988408 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd8300afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd834ed2700 1 -- 172.21.15.7:0/953988408 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414372099 0 0) 0x7fd8300abba0 con 0x7fd830154bc0 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.023+0000 7fd834ed2700 1 -- 172.21.15.7:0/953988408 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003570 con 0x7fd830154bc0 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd8356d3700 1 -- 172.21.15.7:0/953988408 >> v1:172.21.15.50:6789/0 conn(0x7fd8301589e0 legacy=0x7fd8300ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd8356d3700 1 -- 172.21.15.7:0/953988408 >> v1:172.21.15.7:6789/0 conn(0x7fd830154bc0 legacy=0x7fd830154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd8356d3700 1 -- 172.21.15.7:0/953988408 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300ab550 con 0x7fd8300af780 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 Processor -- start 2023-10-06T11:22:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd8356d3700 1 --2- 172.21.15.7:0/953988408 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd8300afbd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e90 tx=0x7fd828009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d26194a4bfd764 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 -- start start 2023-10-06T11:22:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd834ed2700 1 -- 172.21.15.7:0/953988408 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800e070 con 0x7fd8300af780 2023-10-06T11:22:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd834ed2700 1 -- 172.21.15.7:0/953988408 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800b920 con 0x7fd8300af780 2023-10-06T11:22:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd834ed2700 1 -- 172.21.15.7:0/953988408 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c001470 con 0x7fd8300af780 2023-10-06T11:22:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 Processor -- start 2023-10-06T11:22:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 -- start start 2023-10-06T11:22:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 0x7f543c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f543c0ab790 0x7f543c15c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c0abc60 con 0x7f543c159060 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c15d7b0 con 0x7f543c0ab790 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f5440d35700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c0ad240 con 0x7f543c158c90 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543b7fe700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 0x7f543c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f543c0ab790 0x7f543c15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f543c0ab790 0x7f543c15c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41676/0 (socket says 172.21.15.7:41676) 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 -- 172.21.15.7:0/586573539 learned_addr learned my addr 172.21.15.7:0/586573539 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543b7fe700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 0x7f543c0acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57806/0 (socket says 172.21.15.7:57806) 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f343806aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380c6b60 con 0x7f343806db70 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380c2ea0 con 0x7f343815d460 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343ff8f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343806e860 con 0x7f34380cac30 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f343806db70 0x7f34380c7600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54878/0 (socket says 172.21.15.7:54878) 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 -- 172.21.15.7:0/1957152517 learned_addr learned my addr 172.21.15.7:0/1957152517 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343d52a700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f343806aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41803574 0 0) 0x7f34380c6b60 con 0x7f343806db70 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424003570 con 0x7f343806db70 2023-10-06T11:22:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2061407976 0 0) 0x7f3424003570 con 0x7f343806db70 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 msgr2=0x7f343806aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f343806aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 >> v1:172.21.15.50:6789/0 conn(0x7f343815d460 legacy=0x7f34380ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343806e2a0 con 0x7f343806db70 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f3428001470 con 0x7f343806db70 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 >> v1:172.21.15.7:6789/0 conn(0x7f343806db70 legacy=0x7f34380c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f3424003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3424003fd0 0x7f3424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3424004d50 0x7f34240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343806e860 con 0x7f3424004d50 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342400bba0 con 0x7f3424003fd0 2023-10-06T11:22:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342400bd60 con 0x7f34380cac30 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fa2ac554700 1 Processor -- start 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fa2ac554700 1 -- start start 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 -- 172.21.15.7:0/953988408 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 msgr2=0x7fd8300afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 --2- 172.21.15.7:0/953988408 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd8300afbd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e90 tx=0x7fd828009780 comp rx=0 tx=0).stop 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 -- 172.21.15.7:0/953988408 shutdown_connections 2023-10-06T11:22:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 --2- 172.21.15.7:0/953988408 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd8300afbd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 -- 172.21.15.7:0/953988408 >> 172.21.15.7:0/953988408 conn(0x7fd830068900 msgr2=0x7fd8300cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 -- 172.21.15.7:0/953988408 shutdown_connections 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7fd836ed6700 1 -- 172.21.15.7:0/953988408 wait complete. 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 Processor -- start 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 -- start start 2023-10-06T11:22:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 -- 172.21.15.7:0/586573539 >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 msgr2=0x7f543c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 --2- 172.21.15.7:0/586573539 >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 0x7f543c0acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 -- 172.21.15.7:0/586573539 >> v1:172.21.15.50:6789/0 conn(0x7f543c159060 legacy=0x7f543c0ab080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 -- 172.21.15.7:0/586573539 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0abf40 con 0x7f543c0ab790 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f543bfff700 1 --2- 172.21.15.7:0/586573539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c0ab790 0x7f543c15c940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f543c06e8e0 tx=0x7f5434009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c260b68fb2ecf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f543a7fc700 1 -- 172.21.15.7:0/586573539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543400e070 con 0x7f543c0ab790 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3424004d50 0x7f34240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343d52a700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3424003fd0 0x7f3424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343dd2b700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f3424003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 -- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 msgr2=0x7f3424003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f34380cac30 0x7f3424003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 -- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3424003fd0 msgr2=0x7f3424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3424003fd0 0x7f3424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f343e52c700 1 -- 172.21.15.7:0/1957152517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424003570 con 0x7f3424004d50 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7b6c0cc120 0x7f7b6c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c158360 con 0x7f7b6c0b0f80 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0caa50 con 0x7f7b6c0cc120 2023-10-06T11:22:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b71f01700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0b1400 con 0x7f7b6c1569b0 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b70eff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b70eff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43486/0 (socket says 172.21.15.7:43486) 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b70eff700 1 -- 172.21.15.7:0/2092132437 learned_addr learned my addr 172.21.15.7:0/2092132437 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.027+0000 7f7b6bfff700 1 --2- 172.21.15.7:0/2092132437 >> v2:172.21.15.50:3300/0 conn(0x7f7b6c0cc120 0x7f7b6c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f543a7fc700 1 -- 172.21.15.7:0/586573539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543400ba00 con 0x7f543c0ab790 2023-10-06T11:22:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f543a7fc700 1 -- 172.21.15.7:0/586573539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340092e0 con 0x7f543c0ab790 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa2a40b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa2a40ae870 0x7fa2a4159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40af1f0 con 0x7fa2a4156c10 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a40afce0 con 0x7fa2a40ae870 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415cc70 con 0x7fa2a4156840 2023-10-06T11:22:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aaaf1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2a4156c10 0x7fa2a415d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54882/0 (socket says 172.21.15.7:54882) 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aaaf1700 1 -- 172.21.15.7:0/4264863086 learned_addr learned my addr 172.21.15.7:0/4264863086 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aa2f0700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa2a40b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832795581 0 0) 0x7fa2a40af1f0 con 0x7fa2a4156c10 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b70eff700 1 -- 172.21.15.7:0/2092132437 >> v2:172.21.15.50:3300/0 conn(0x7f7b6c0cc120 msgr2=0x7f7b6c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b70eff700 1 --2- 172.21.15.7:0/2092132437 >> v2:172.21.15.50:3300/0 conn(0x7f7b6c0cc120 0x7f7b6c0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa290003570 con 0x7fa2a4156c10 2023-10-06T11:22:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b70eff700 1 -- 172.21.15.7:0/2092132437 >> v1:172.21.15.50:6789/0 conn(0x7f7b6c0b0f80 legacy=0x7f7b6c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b70eff700 1 -- 172.21.15.7:0/2092132437 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0c8b80 con 0x7f7b6c1569b0 2023-10-06T11:22:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343e52c700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3424004d50 0x7f34240051c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34280050b0 tx=0x7f3428008d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e574bc29101208e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168378792 0 0) 0x7fa290003570 con 0x7fa2a4156c10 2023-10-06T11:22:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 msgr2=0x7fa2a40b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa2a40b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> v2:172.21.15.50:3300/0 conn(0x7fa2a40ae870 0x7fa2a4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 >> v2:172.21.15.50:3300/0 conn(0x7fa2a40ae870 msgr2=0x7fa2a4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fd836ed6700 1 Processor -- start 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 --2- 172.21.15.7:0/4264863086 >> v2:172.21.15.50:3300/0 conn(0x7fa2a40ae870 0x7fa2a4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40af440 con 0x7fa2a4156c10 2023-10-06T11:22:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fd836ed6700 1 -- start start 2023-10-06T11:22:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b70eff700 1 --2- 172.21.15.7:0/2092132437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0099b0 tx=0x7f7b5c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdf810197a3369 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7fa294001470 con 0x7fa2a4156c10 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 >> v1:172.21.15.7:6789/0 conn(0x7fa2a4156c10 legacy=0x7fa2a415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa290003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900040b0 0x7fa290004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa290004da0 0x7fa290005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280049f0 con 0x7f3424004d50 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3428005920 con 0x7f3424004d50 2023-10-06T11:22:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343cd29700 1 -- 172.21.15.7:0/1957152517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428005d00 con 0x7f3424004d50 2023-10-06T11:22:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40afce0 con 0x7fa2900040b0 2023-10-06T11:22:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415cc70 con 0x7fa290004da0 2023-10-06T11:22:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29000bcf0 con 0x7fa2a4156840 2023-10-06T11:22:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 Processor -- start 2023-10-06T11:22:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 -- start start 2023-10-06T11:22:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> v2:172.21.15.50:3300/0 conn(0x7fa2a40ae870 0x7fa2a4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 -- 172.21.15.7:0/586573539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c0ab790 msgr2=0x7f543c15c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 --2- 172.21.15.7:0/586573539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c0ab790 0x7f543c15c940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f543c06e8e0 tx=0x7f5434009840 comp rx=0 tx=0).stop 2023-10-06T11:22:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b6b7fe700 1 -- 172.21.15.7:0/2092132437 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c009ea0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900040b0 0x7fa290004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aa2f0700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa290003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aaaf1700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa290004da0 0x7fa290005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 -- 172.21.15.7:0/586573539 shutdown_connections 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 --2- 172.21.15.7:0/586573539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c0ab790 0x7f543c15c940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 --2- 172.21.15.7:0/586573539 >> v2:172.21.15.50:3300/0 conn(0x7f543c158c90 0x7f543c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 -- 172.21.15.7:0/586573539 >> 172.21.15.7:0/586573539 conn(0x7f543c068900 msgr2=0x7f543c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b6b7fe700 1 -- 172.21.15.7:0/2092132437 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c00b860 con 0x7f7b6c1569b0 2023-10-06T11:22:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b6b7fe700 1 -- 172.21.15.7:0/2092132437 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00bc00 con 0x7f7b6c1569b0 2023-10-06T11:22:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 Processor -- start 2023-10-06T11:22:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 -- start start 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 -- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 msgr2=0x7fa290003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a4156840 0x7fa290003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 -- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa290004da0 msgr2=0x7fa290005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa290004da0 0x7fa290005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 -- 172.21.15.7:0/4264863086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa290003570 con 0x7fa2900040b0 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aaaf1700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa290004da0 0x7fa290005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a9aef700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900040b0 0x7fa290004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000abe0 tx=0x7fa2a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7199626253b094be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 -- 172.21.15.7:0/1957152517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3424004d50 msgr2=0x7f34240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 --2- 172.21.15.7:0/1957152517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3424004d50 0x7f34240051c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34280050b0 tx=0x7f3428008d30 comp rx=0 tx=0).stop 2023-10-06T11:22:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343e52c700 1 -- 172.21.15.7:0/1957152517 reap_dead start 2023-10-06T11:22:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 -- 172.21.15.7:0/1957152517 shutdown_connections 2023-10-06T11:22:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 -- 172.21.15.7:0/1957152517 >> 172.21.15.7:0/1957152517 conn(0x7f34380688c0 msgr2=0x7f34380cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f81840ca140 0x7f81840ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 -- 172.21.15.7:0/1957152517 shutdown_connections 2023-10-06T11:22:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000c6d0 con 0x7fa2900040b0 2023-10-06T11:22:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2a92ee700 1 -- 172.21.15.7:0/4264863086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000c830 con 0x7fa2900040b0 2023-10-06T11:22:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8184069f50 con 0x7f818406afd0 2023-10-06T11:22:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818406a230 con 0x7f81840c8bb0 2023-10-06T11:22:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8188cc4700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840c9030 con 0x7f81840ca140 2023-10-06T11:22:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8181d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f818406afd0 0x7f81840c84a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54886/0 (socket says 172.21.15.7:54886) 2023-10-06T11:22:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f8181d9b700 1 -- 172.21.15.7:0/2333560151 learned_addr learned my addr 172.21.15.7:0/2333560151 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 0x7f79840ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840aba80 con 0x7f7984155070 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79840af7f0 con 0x7f79840b02a0 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7983fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840acf20 con 0x7f79840c8610 2023-10-06T11:22:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f5440d35700 1 -- 172.21.15.7:0/586573539 shutdown_connections 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f79837fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f79837fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43500/0 (socket says 172.21.15.7:43500) 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f79837fe700 1 -- 172.21.15.7:0/3964875711 learned_addr learned my addr 172.21.15.7:0/3964875711 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f79827fc700 1 --2- 172.21.15.7:0/3964875711 >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 0x7f79840ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f818259c700 1 --2- 172.21.15.7:0/2333560151 >> v2:172.21.15.50:3300/0 conn(0x7f81840ca140 0x7f81840ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fd836ed6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd830164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b71f01700 1 -- 172.21.15.7:0/2092132437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 msgr2=0x7f7b6c0c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b71f01700 1 --2- 172.21.15.7:0/2092132437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0099b0 tx=0x7f7b5c009250 comp rx=0 tx=0).stop 2023-10-06T11:22:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 -- 172.21.15.7:0/4264863086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900040b0 msgr2=0x7fa290004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2ac554700 1 --2- 172.21.15.7:0/4264863086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900040b0 0x7fa290004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000abe0 tx=0x7fa2a0008040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7fa2aaaf1700 1 -- 172.21.15.7:0/4264863086 reap_dead start 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b71f01700 1 -- 172.21.15.7:0/2092132437 shutdown_connections 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b71f01700 1 --2- 172.21.15.7:0/2092132437 >> v2:172.21.15.50:3300/0 conn(0x7f7b6c0cc120 0x7f7b6c0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f7b71f01700 1 --2- 172.21.15.7:0/2092132437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c1569b0 0x7f7b6c0c8330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7b71f01700 1 -- 172.21.15.7:0/2092132437 >> 172.21.15.7:0/2092132437 conn(0x7f7b6c068900 msgr2=0x7f7b6c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.031+0000 7f343ff8f700 1 -- 172.21.15.7:0/1957152517 wait complete. 2023-10-06T11:22:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- 172.21.15.7:0/4264863086 shutdown_connections 2023-10-06T11:22:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- 172.21.15.7:0/4264863086 >> 172.21.15.7:0/4264863086 conn(0x7fa2a4066840 msgr2=0x7fa2a4066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7b71f01700 1 -- 172.21.15.7:0/2092132437 shutdown_connections 2023-10-06T11:22:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8301589e0 0x7fd830164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830165f60 con 0x7fd830154bc0 2023-10-06T11:22:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818159a700 1 -- 172.21.15.7:0/2333560151 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334245177 0 0) 0x7f8184069f50 con 0x7f818406afd0 2023-10-06T11:22:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd835ed4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd830164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- 172.21.15.7:0/4264863086 shutdown_connections 2023-10-06T11:22:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41716/0 (socket says 172.21.15.7:41716) 2023-10-06T11:22:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd835ed4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd830164540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43504/0 (socket says 172.21.15.7:43504) 2023-10-06T11:22:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 -- 172.21.15.7:0/1806277690 learned_addr learned my addr 172.21.15.7:0/1806277690 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 -- 172.21.15.7:0/586573539 wait complete. 2023-10-06T11:22:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79837fe700 1 -- 172.21.15.7:0/3964875711 >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 msgr2=0x7f79840ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818159a700 1 -- 172.21.15.7:0/2333560151 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c003570 con 0x7f818406afd0 2023-10-06T11:22:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300c8870 con 0x7fd8301589e0 2023-10-06T11:22:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300c8b10 con 0x7fd8300af780 2023-10-06T11:22:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- 172.21.15.7:0/4264863086 wait complete. 2023-10-06T11:22:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8366d5700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8301589e0 0x7fd830164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7981ffb700 1 -- 172.21.15.7:0/3964875711 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452470326 0 0) 0x7f79840aba80 con 0x7f7984155070 2023-10-06T11:22:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79837fe700 1 --2- 172.21.15.7:0/3964875711 >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 0x7f79840ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79837fe700 1 -- 172.21.15.7:0/3964875711 >> v1:172.21.15.7:6789/0 conn(0x7f7984155070 legacy=0x7f79840afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79837fe700 1 -- 172.21.15.7:0/3964875711 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840af510 con 0x7f79840c8610 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79827fc700 1 --2- 172.21.15.7:0/3964875711 >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 0x7f79840ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 Processor -- start 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818259c700 1 -- 172.21.15.7:0/2333560151 >> v1:172.21.15.50:6789/0 conn(0x7f81840c8bb0 legacy=0x7f81840c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818259c700 1 -- 172.21.15.7:0/2333560151 >> v1:172.21.15.7:6789/0 conn(0x7f818406afd0 legacy=0x7f81840c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818259c700 1 -- 172.21.15.7:0/2333560151 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818406a510 con 0x7f81840ca140 2023-10-06T11:22:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 -- start start 2023-10-06T11:22:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 -- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 msgr2=0x7fd830164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd830164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 -- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8301589e0 msgr2=0x7fd830164fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8301589e0 0x7fd830164fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f79837fe700 1 --2- 172.21.15.7:0/3964875711 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f797c00d1a0 tx=0x7f797c00ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec38474514f4bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009a20 con 0x7fd830154bc0 2023-10-06T11:22:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818259c700 1 --2- 172.21.15.7:0/2333560151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81840ca140 0x7f81840ca510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f817c009e60 tx=0x7f817c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634209627be5cc08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7981ffb700 1 -- 172.21.15.7:0/3964875711 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c010070 con 0x7f79840c8610 2023-10-06T11:22:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd8356d3700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd830068d70 tx=0x7fd828017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818159a700 1 -- 172.21.15.7:0/2333560151 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00e070 con 0x7f81840ca140 2023-10-06T11:22:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7981ffb700 1 -- 172.21.15.7:0/3964875711 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00dc00 con 0x7f79840c8610 2023-10-06T11:22:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7981ffb700 1 -- 172.21.15.7:0/3964875711 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c016400 con 0x7f79840c8610 2023-10-06T11:22:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818159a700 1 -- 172.21.15.7:0/2333560151 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00bb60 con 0x7f81840ca140 2023-10-06T11:22:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f818159a700 1 -- 172.21.15.7:0/2333560151 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c012430 con 0x7f81840ca140 2023-10-06T11:22:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 Processor -- start 2023-10-06T11:22:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 -- start start 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828009420 con 0x7fd830154bc0 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828012b30 con 0x7fd830154bc0 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82801e450 con 0x7fd830154bc0 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8301656c0 con 0x7fd830154bc0 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830165c80 con 0x7fd830154bc0 2023-10-06T11:22:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 Processor -- start 2023-10-06T11:22:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34381619e0 0x7f34380b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34380b11a0 0x7f34380c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438160470 con 0x7f34381619e0 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381606e0 con 0x7f34380b11a0 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343ff8f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34381609c0 con 0x7f343815aa00 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43512/0 (socket says 172.21.15.7:43512) 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 -- 172.21.15.7:0/3991152941 learned_addr learned my addr 172.21.15.7:0/3991152941 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343e52c700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34380b11a0 0x7f34380c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 -- 172.21.15.7:0/2333560151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81840ca140 msgr2=0x7f81840ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 --2- 172.21.15.7:0/2333560151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81840ca140 0x7f81840ca510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f817c009e60 tx=0x7f817c009780 comp rx=0 tx=0).stop 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 -- 172.21.15.7:0/2333560151 shutdown_connections 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 --2- 172.21.15.7:0/2333560151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81840ca140 0x7f81840ca510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 -- 172.21.15.7:0/2333560151 >> 172.21.15.7:0/2333560151 conn(0x7f8184067e30 msgr2=0x7f8184068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 -- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34380b11a0 msgr2=0x7f34380c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34380b11a0 0x7f34380c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 -- 172.21.15.7:0/3991152941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34381619e0 msgr2=0x7f34380b0c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:22:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34381619e0 0x7f34380b0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 -- 172.21.15.7:0/3991152941 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438160ca0 con 0x7f343815aa00 2023-10-06T11:22:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- start start 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f543c0ab790 0x7f543c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f543c159060 0x7f543c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c159d80 con 0x7f543c158c90 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15a620 con 0x7f543c0ab790 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f5440d35700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0a93c0 con 0x7f543c159060 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f543affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f543affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41720/0 (socket says 172.21.15.7:41720) 2023-10-06T11:22:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f543affd700 1 -- 172.21.15.7:0/3402552402 learned_addr learned my addr 172.21.15.7:0/3402552402 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 0x7fa2a4152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a415ab30 0x7fa2a4152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 0x7fa2a40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 -- 172.21.15.7:0/2333560151 shutdown_connections 2023-10-06T11:22:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f343dd2b700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3434002920 tx=0x7f3434007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a4168ce0 con 0x7fa2a414fcd0 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a4168fb0 con 0x7fa2a415ab30 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2ac554700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a4169290 con 0x7fa2a414f900 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2aa2f0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 0x7fa2a4152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2aa2f0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 0x7fa2a4152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43528/0 (socket says 172.21.15.7:43528) 2023-10-06T11:22:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aa2f0700 1 -- 172.21.15.7:0/2162990157 learned_addr learned my addr 172.21.15.7:0/2162990157 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7fa2aaaf1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 0x7fa2a40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400f720 con 0x7f343815aa00 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 -- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f543c159060 msgr2=0x7f543c14f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f543c159060 0x7f543c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 -- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f543c0ab790 msgr2=0x7f543c15bd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f543c0ab790 0x7f543c15bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 -- 172.21.15.7:0/3402552402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434009ae0 con 0x7f543c158c90 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343400f880 con 0x7f343815aa00 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400fc40 con 0x7f343815aa00 2023-10-06T11:22:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f343ff8f700 1 -- 172.21.15.7:0/3991152941 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3438160f80 con 0x7f343815aa00 2023-10-06T11:22:19.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f343ff8f700 1 -- 172.21.15.7:0/3991152941 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343815add0 con 0x7f343815aa00 2023-10-06T11:22:19.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f543affd700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f543000b5e0 tx=0x7f54300090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd82801e880 con 0x7fd830154bc0 2023-10-06T11:22:19.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd825ffb700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 0x7fd814062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f7b71f01700 1 -- 172.21.15.7:0/2092132437 wait complete. 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 Processor -- start 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 -- start start 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 -- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 msgr2=0x7fa2a4152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 0x7fa2a4152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 -- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a415ab30 msgr2=0x7fa2a4152d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a415ab30 0x7fa2a4152d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 -- 172.21.15.7:0/2162990157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4169570 con 0x7fa2a414fcd0 2023-10-06T11:22:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.035+0000 7f8188cc4700 1 -- 172.21.15.7:0/2333560151 wait complete. 2023-10-06T11:22:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2aaaf1700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 0x7fa2a40c7cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2a406d720 tx=0x7fa2a415cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 Processor -- start 2023-10-06T11:22:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f343ff8f700 1 -- 172.21.15.7:0/3991152941 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3438070f20 con 0x7f343815aa00 2023-10-06T11:22:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd828052770 con 0x7fd830154bc0 2023-10-06T11:22:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 -- 172.21.15.7:0/3964875711 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 msgr2=0x7f79840aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 --2- 172.21.15.7:0/3964875711 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f797c00d1a0 tx=0x7f797c00ba50 comp rx=0 tx=0).stop 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 Processor -- start 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 -- start start 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000f510 con 0x7f543c158c90 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000f690 con 0x7f543c158c90 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000fa50 con 0x7f543c158c90 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f5440d35700 1 -- 172.21.15.7:0/3402552402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c159fd0 con 0x7f543c158c90 2023-10-06T11:22:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f5440d35700 1 -- 172.21.15.7:0/3402552402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c0a90b0 con 0x7f543c158c90 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f5440d35700 1 -- 172.21.15.7:0/3402552402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c06c710 con 0x7f543c158c90 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 -- start start 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f818406afd0 0x7f818415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 0x7f818414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184172330 con 0x7f81840ca140 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840b1720 con 0x7f818406afd0 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8188cc4700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818416e030 con 0x7f81840c8bb0 2023-10-06T11:22:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43534/0 (socket says 172.21.15.7:43534) 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 -- 172.21.15.7:0/264043526 learned_addr learned my addr 172.21.15.7:0/264043526 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8182d9d700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 0x7f818414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f818259c700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f818406afd0 0x7f818415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 -- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f818406afd0 msgr2=0x7f818415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f818406afd0 0x7f818415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 -- 172.21.15.7:0/264043526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 msgr2=0x7f818414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 0x7f818414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8181d9b700 1 -- 172.21.15.7:0/264043526 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c009a20 con 0x7f81840c8bb0 2023-10-06T11:22:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8182d9d700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 0x7f818414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8100052b0 con 0x7fd830154bc0 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd835ed4700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 0x7fd814062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fd835ed4700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 0x7fd814062a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0011e0 tx=0x7fd82c00a0f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 -- 172.21.15.7:0/3964875711 shutdown_connections 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 --2- 172.21.15.7:0/3964875711 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f79840aece0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 --2- 172.21.15.7:0/3964875711 >> v2:172.21.15.7:3300/0 conn(0x7f79840b02a0 0x7f79840ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 -- 172.21.15.7:0/3964875711 >> 172.21.15.7:0/3964875711 conn(0x7f79840688a0 msgr2=0x7f79840cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 -- 172.21.15.7:0/3964875711 shutdown_connections 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7983fff700 1 -- 172.21.15.7:0/3964875711 wait complete. 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 0x7f7b6c175e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b6c0cc120 0x7f7b6c178cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c15b8f0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1763d0 con 0x7f7b6c0cc120 2023-10-06T11:22:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71f01700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c179200 con 0x7f7b6c0b0f80 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b70eff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 0x7f7b6c175e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41730/0 (socket says 172.21.15.7:41730) 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 -- 172.21.15.7:0/930304502 learned_addr learned my addr 172.21.15.7:0/930304502 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 -- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 msgr2=0x7f7b6c175e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 0x7f7b6c175e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 -- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b6c0cc120 msgr2=0x7f7b6c178cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b6c0cc120 0x7f7b6c178cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 -- 172.21.15.7:0/930304502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0094f0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b70eff700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 0x7f7b6c175e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f7b71700700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c4f0 tx=0x7f7b60007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40c8ef0 0x7f8dd415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8dd40c92c0 0x7f8dd40b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd4158220 con 0x7f8dd406ab30 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd415acb0 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd9642700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd41579b0 con 0x7f8dd40c92c0 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7f8dd2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40c8ef0 0x7f8dd415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40c8ef0 0x7f8dd415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41746/0 (socket says 172.21.15.7:41746) 2023-10-06T11:22:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/2064229659 learned_addr learned my addr 172.21.15.7:0/2064229659 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8181d9b700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f817800c790 tx=0x7f8178007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940114f0 con 0x7fa2a414fcd0 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294011670 con 0x7fa2a414fcd0 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011a30 con 0x7fa2a414fcd0 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a4169850 con 0x7fa2a414fcd0 2023-10-06T11:22:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.039+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a40c81f0 con 0x7fa2a414fcd0 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/2064229659 >> v2:172.21.15.50:3300/0 conn(0x7f8dd40c92c0 msgr2=0x7f8dd40b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 --2- 172.21.15.7:0/2064229659 >> v2:172.21.15.50:3300/0 conn(0x7f8dd40c92c0 0x7f8dd40b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/2064229659 >> v1:172.21.15.7:6789/0 conn(0x7f8dd406ab30 legacy=0x7f8dd41572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/2064229659 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4157c60 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f8dd2ffd700 1 --2- 172.21.15.7:0/2064229659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40c8ef0 0x7f8dd415b140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40cd2c0 tx=0x7f8dcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0af772adcc201b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3434025430 con 0x7f343815aa00 2023-10-06T11:22:19.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f543001a400 con 0x7f543c158c90 2023-10-06T11:22:19.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f54237fe700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 0x7f5428062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f543007c660 con 0x7f543c158c90 2023-10-06T11:22:19.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f543b7fe700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 0x7f5428062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 Processor -- start 2023-10-06T11:22:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 -- start start 2023-10-06T11:22:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f342dffb700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 0x7f341c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f343407d180 con 0x7f343815aa00 2023-10-06T11:22:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f543b7fe700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 0x7f5428062ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f542c009a60 tx=0x7f542c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f343d52a700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 0x7f341c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 0x7f79841505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f7984165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7984150ae0 con 0x7f7984155070 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840aaa20 con 0x7f79840b02a0 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7983fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840a4980 con 0x7f79840c8610 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41754/0 (socket says 172.21.15.7:41754) 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 -- 172.21.15.7:0/3783346289 learned_addr learned my addr 172.21.15.7:0/3783346289 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7982ffd700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 0x7f79841505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79827fc700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f7984165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f343d52a700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 0x7f341c062a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f343812d650 tx=0x7f343000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 -- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 msgr2=0x7f7984165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f7984165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 -- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 msgr2=0x7f79841505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 0x7f79841505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 -- 172.21.15.7:0/3783346289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978009420 con 0x7f7984155070 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79827fc700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f7984165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f7982ffd700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 0x7f79841505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.043+0000 7f79837fe700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f797c016000 tx=0x7f797c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011590 con 0x7f7b6c1569b0 2023-10-06T11:22:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f8dd1ffb700 1 -- 172.21.15.7:0/2064229659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00e070 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c15ace0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c15b2a0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c02a410 con 0x7f7984155070 2023-10-06T11:22:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd828083060 con 0x7fd830154bc0 2023-10-06T11:22:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c00bcf0 con 0x7f7984155070 2023-10-06T11:22:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.047+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79840a4f40 con 0x7f7984155070 2023-10-06T11:22:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780072e0 con 0x7f81840c8bb0 2023-10-06T11:22:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8178007440 con 0x7f81840c8bb0 2023-10-06T11:22:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818416dd40 con 0x7f81840c8bb0 2023-10-06T11:22:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f343404d3a0 con 0x7f343815aa00 2023-10-06T11:22:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780057f0 con 0x7f81840c8bb0 2023-10-06T11:22:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8184171d70 con 0x7f81840c8bb0 2023-10-06T11:22:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c0052b0 con 0x7fa2a414fcd0 2023-10-06T11:22:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c15baa0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79841554d0 con 0x7f7984155070 2023-10-06T11:22:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f54300503f0 con 0x7f543c158c90 2023-10-06T11:22:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8178005ba0 con 0x7f81840c8bb0 2023-10-06T11:22:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f8dd1ffb700 1 -- 172.21.15.7:0/2064229659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00ba50 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f8dd1ffb700 1 -- 172.21.15.7:0/2064229659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0092e0 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 0x7f8164062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 -- 172.21.15.7:0/2064229659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40c8ef0 msgr2=0x7f8dd415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 --2- 172.21.15.7:0/2064229659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40c8ef0 0x7f8dd415b140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40cd2c0 tx=0x7f8dcc009840 comp rx=0 tx=0).stop 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 -- 172.21.15.7:0/2064229659 shutdown_connections 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 --2- 172.21.15.7:0/2064229659 >> v2:172.21.15.50:3300/0 conn(0x7f8dd40c92c0 0x7f8dd40b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 --2- 172.21.15.7:0/2064229659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40c8ef0 0x7f8dd415b140 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8dd9642700 1 -- 172.21.15.7:0/2064229659 >> 172.21.15.7:0/2064229659 conn(0x7f8dd4067e30 msgr2=0x7f8dd4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f817807bfc0 con 0x7f81840c8bb0 2023-10-06T11:22:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f818259c700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 0x7f8164062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f818259c700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 0x7f8164062ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f817c0097c0 tx=0x7f817c019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa2940133e0 con 0x7fa2a414fcd0 2023-10-06T11:22:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b600010b0 con 0x7f7b6c1569b0 2023-10-06T11:22:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.051+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011900 con 0x7f7b6c1569b0 2023-10-06T11:22:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7b60011b20 con 0x7f7b6c1569b0 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f7b68ff9700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 0x7f7b58062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f7b70eff700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 0x7f7b58062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f7b70eff700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 0x7f7b58062a30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009290 tx=0x7f7b5c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c02a780 con 0x7f7984155070 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c02a8f0 con 0x7f7984155070 2023-10-06T11:22:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f797c020410 con 0x7f7984155070 2023-10-06T11:22:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7fa29a7fc700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 0x7fa288062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa29407c620 con 0x7fa2a414fcd0 2023-10-06T11:22:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f796affd700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 0x7f7970062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f7982ffd700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 0x7f7970062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7fa2aa2f0700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 0x7fa288062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.055+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184070f20 con 0x7f81840c8bb0 2023-10-06T11:22:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- 172.21.15.7:0/2064229659 shutdown_connections 2023-10-06T11:22:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- 172.21.15.7:0/2064229659 wait complete. 2023-10-06T11:22:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7fa2aa2f0700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 0x7fa288062ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40c4f90 tx=0x7fa29c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f7982ffd700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 0x7f7970062a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7974007fd0 tx=0x7f7974009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 Processor -- start 2023-10-06T11:22:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- start start 2023-10-06T11:22:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 0x7f8dd40a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dd40c92c0 0x7f8dd40af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41696b0 con 0x7f8dd406ab30 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd414f2e0 con 0x7f8dd40c92c0 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd9642700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41576c0 con 0x7f8dd40c8ef0 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41760/0 (socket says 172.21.15.7:41760) 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/1035133815 learned_addr learned my addr 172.21.15.7:0/1035133815 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd37fe700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dd40c92c0 0x7f8dd40af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 msgr2=0x7f8dd40a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd27fc700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 0x7f8dd40a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 0x7f8dd40a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dd40c92c0 msgr2=0x7f8dd40af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dd40c92c0 0x7f8dd40af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 -- 172.21.15.7:0/1035133815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc009ae0 con 0x7f8dd406ab30 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd27fc700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 0x7f8dd40a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.059+0000 7f8dd2ffd700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009fd0 tx=0x7f8dcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.063+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8178049be0 con 0x7f81840c8bb0 2023-10-06T11:22:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.063+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7b60053430 con 0x7f7b6c1569b0 2023-10-06T11:22:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.063+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa29404c920 con 0x7fa2a414fcd0 2023-10-06T11:22:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.063+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f797c051210 con 0x7f7984155070 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.067+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc008a60 con 0x7f8dd406ab30 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.067+0000 7f8dd9642700 1 -- 172.21.15.7:0/1035133815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd4156e20 con 0x7f8dd406ab30 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.067+0000 7f8dd9642700 1 -- 172.21.15.7:0/1035133815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd41573e0 con 0x7f8dd406ab30 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.071+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4070f20 con 0x7f8dd406ab30 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.071+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b6004c590 con 0x7f7b6c1569b0 2023-10-06T11:22:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.079+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f797c054c80 con 0x7f7984155070 2023-10-06T11:22:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc008dd0 con 0x7f8dd406ab30 2023-10-06T11:22:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc026460 con 0x7f8dd406ab30 2023-10-06T11:22:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8dcc003040 con 0x7f8dd406ab30 2023-10-06T11:22:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dbaffd700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 0x7f8dc0062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8dcc07ea00 con 0x7f8dd406ab30 2023-10-06T11:22:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dd27fc700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 0x7f8dc0062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.087+0000 7f8dd27fc700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 0x7f8dc0062b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b1ef0 tx=0x7f8dc4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.091+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8dcc04ec00 con 0x7f8dd406ab30 2023-10-06T11:22:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.351+0000 7fd836ed6700 1 -- 172.21.15.7:0/1806277690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd810005830 con 0x7fd830154bc0 2023-10-06T11:22:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.351+0000 7fd825ffb700 1 -- 172.21.15.7:0/1806277690 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd82802b020 con 0x7fd830154bc0 2023-10-06T11:22:19.360 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.355+0000 7f343ff8f700 1 -- 172.21.15.7:0/3991152941 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f343815b0b0 con 0x7f343815aa00 2023-10-06T11:22:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.355+0000 7f342dffb700 1 -- 172.21.15.7:0/3991152941 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34340509c0 con 0x7f343815aa00 2023-10-06T11:22:19.360 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:19.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 msgr2=0x7fd814062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 0x7fd814062a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0011e0 tx=0x7fd82c00a0f0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 msgr2=0x7fd830164a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd830068d70 tx=0x7fd828017000 comp rx=0 tx=0).stop 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 shutdown_connections 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd814060550 0x7fd814062a00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8301589e0 0x7fd830164fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830154bc0 0x7fd830164a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 --2- 172.21.15.7:0/1806277690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd8300af780 0x7fd830164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 >> 172.21.15.7:0/1806277690 conn(0x7fd830068900 msgr2=0x7fd8300ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 shutdown_connections 2023-10-06T11:22:19.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.359+0000 7fd81b7fe700 1 -- 172.21.15.7:0/1806277690 wait complete. 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 msgr2=0x7f341c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 0x7f341c062a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f343812d650 tx=0x7f343000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 msgr2=0x7f34381614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3434002920 tx=0x7f3434007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 shutdown_connections 2023-10-06T11:22:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f341c060580 0x7f341c062a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34380b11a0 0x7f34380c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34381619e0 0x7f34380b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 --2- 172.21.15.7:0/3991152941 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343815aa00 0x7f34381614a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 >> 172.21.15.7:0/3991152941 conn(0x7f34380688c0 msgr2=0x7f34380c0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 shutdown_connections 2023-10-06T11:22:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.367+0000 7f34237fe700 1 -- 172.21.15.7:0/3991152941 wait complete. 2023-10-06T11:22:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.371+0000 7f5440d35700 1 -- 172.21.15.7:0/3402552402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f543c0ac220 con 0x7f543c158c90 2023-10-06T11:22:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.371+0000 7f54237fe700 1 -- 172.21.15.7:0/3402552402 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f543004ff80 con 0x7f543c158c90 2023-10-06T11:22:19.375 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 msgr2=0x7f5428062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 0x7f5428062ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f542c009a60 tx=0x7f542c008040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 msgr2=0x7f543c0a8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f543000b5e0 tx=0x7f54300090f0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 shutdown_connections 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5428060620 0x7f5428062ad0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f543c159060 0x7f543c14f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c158c90 0x7f543c0a8b70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 --2- 172.21.15.7:0/3402552402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f543c0ab790 0x7f543c15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 >> 172.21.15.7:0/3402552402 conn(0x7f543c068900 msgr2=0x7f543c150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 shutdown_connections 2023-10-06T11:22:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.375+0000 7f54217fa700 1 -- 172.21.15.7:0/3402552402 wait complete. 2023-10-06T11:22:19.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-10-06T11:22:19.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-10-06T11:22:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.383+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa28c005830 con 0x7fa2a414fcd0 2023-10-06T11:22:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.383+0000 7fa29a7fc700 1 -- 172.21.15.7:0/2162990157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa294019020 con 0x7fa2a414fcd0 2023-10-06T11:22:19.388 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-06T11:22:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 msgr2=0x7fa288062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 0x7fa288062ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40c4f90 tx=0x7fa29c00b040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 msgr2=0x7fa2a40c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 0x7fa2a40c7cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2a406d720 tx=0x7fa2a415cbe0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 shutdown_connections 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa288060620 0x7fa288062ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a414fcd0 0x7fa2a40c7cb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a415ab30 0x7fa2a4152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 --2- 172.21.15.7:0/2162990157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa2a414f900 0x7fa2a4152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 >> 172.21.15.7:0/2162990157 conn(0x7fa2a4066840 msgr2=0x7fa2a4155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 shutdown_connections 2023-10-06T11:22:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7fa2ac554700 1 -- 172.21.15.7:0/2162990157 wait complete. 2023-10-06T11:22:19.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-10-06T11:22:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b6c15ba30 con 0x7f7b6c1569b0 2023-10-06T11:22:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7f7b68ff9700 1 -- 172.21.15.7:0/930304502 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b60021020 con 0x7f7b6c1569b0 2023-10-06T11:22:19.394 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-06T11:22:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.387+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f818416e170 con 0x7f81840c8bb0 2023-10-06T11:22:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f81727fc700 1 -- 172.21.15.7:0/264043526 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8178017020 con 0x7f81840c8bb0 2023-10-06T11:22:19.395 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 msgr2=0x7f7b58062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 0x7f7b58062a30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009290 tx=0x7f7b5c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 msgr2=0x7f7b6c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c4f0 tx=0x7f7b60007f50 comp rx=0 tx=0).stop 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 shutdown_connections 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b58060580 0x7f7b58062a30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c1569b0 0x7f7b6c15a260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b6c0cc120 0x7f7b6c178cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 --2- 172.21.15.7:0/930304502 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b6c0b0f80 0x7f7b6c175e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 >> 172.21.15.7:0/930304502 conn(0x7f7b6c068900 msgr2=0x7f7b6c0ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 shutdown_connections 2023-10-06T11:22:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f7b71f01700 1 -- 172.21.15.7:0/930304502 wait complete. 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 msgr2=0x7f8164062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 0x7f8164062ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f817c0097c0 tx=0x7f817c019000 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 msgr2=0x7f8184171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f817800c790 tx=0x7f8178007c80 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 shutdown_connections 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8164060620 0x7f8164062ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840ca140 0x7f818414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f81840c8bb0 0x7f8184171390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 --2- 172.21.15.7:0/264043526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f818406afd0 0x7f818415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 >> 172.21.15.7:0/264043526 conn(0x7f8184067e30 msgr2=0x7f8184154410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 shutdown_connections 2023-10-06T11:22:19.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.391+0000 7f8188cc4700 1 -- 172.21.15.7:0/264043526 wait complete. 2023-10-06T11:22:19.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.399+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f79841554d0 con 0x7f7984155070 2023-10-06T11:22:19.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.399+0000 7f796affd700 1 -- 172.21.15.7:0/3783346289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79841554d0 con 0x7f7984155070 2023-10-06T11:22:19.404 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-06T11:22:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.399+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 msgr2=0x7f7970062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.399+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 0x7f7970062a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7974007fd0 tx=0x7f7974009040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 msgr2=0x7f79840a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f797c016000 tx=0x7f797c01c000 comp rx=0 tx=0).stop 2023-10-06T11:22:19.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 shutdown_connections 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79700605d0 0x7f7970062a80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7984155070 0x7f79840a5cb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79840c8610 0x7f7984165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 --2- 172.21.15.7:0/3783346289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79840b02a0 0x7f79841505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 >> 172.21.15.7:0/3783346289 conn(0x7f79840688a0 msgr2=0x7f7984153250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 shutdown_connections 2023-10-06T11:22:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f7983fff700 1 -- 172.21.15.7:0/3783346289 wait complete. 2023-10-06T11:22:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8dd414f450 con 0x7f8dd406ab30 2023-10-06T11:22:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8dbaffd700 1 -- 172.21.15.7:0/1035133815 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dcc052220 con 0x7f8dd406ab30 2023-10-06T11:22:19.409 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:19.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 msgr2=0x7f8dc0062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 0x7f8dc0062b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b1ef0 tx=0x7f8dc4009040 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 msgr2=0x7f8dd414eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009fd0 tx=0x7f8dcc018000 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 shutdown_connections 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8dc0060650 0x7f8dc0062b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dd40c92c0 0x7f8dd40af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8dd40c8ef0 0x7f8dd40a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 --2- 172.21.15.7:0/1035133815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd406ab30 0x7f8dd414eda0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 >> 172.21.15.7:0/1035133815 conn(0x7f8dd4067e30 msgr2=0x7f8dd40ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:19.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 shutdown_connections 2023-10-06T11:22:19.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:19.403+0000 7f8db8ff9700 1 -- 172.21.15.7:0/1035133815 wait complete. 2023-10-06T11:22:19.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-10-06T11:22:19.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-10-06T11:22:19.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-10-06T11:22:20.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:22:20.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:22:20.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:22:20.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:22:20.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:22:20.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:22:20.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:22:20.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:22:20.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.507+0000 7f633dc35700 1 Processor -- start 2023-10-06T11:22:20.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.507+0000 7f633dc35700 1 -- start start 2023-10-06T11:22:20.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f633dc35700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63380cc450 0x7f63380af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f633dc35700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633806dca0 con 0x7f6338152ac0 2023-10-06T11:22:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f633dc35700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6338152e90 con 0x7f63380ae7c0 2023-10-06T11:22:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f633dc35700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380b0180 con 0x7f63380cc450 2023-10-06T11:22:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f63377fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6338152ac0 0x7f63380cbd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54910/0 (socket says 172.21.15.7:54910) 2023-10-06T11:22:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f63377fe700 1 -- 172.21.15.7:0/3635106377 learned_addr learned my addr 172.21.15.7:0/3635106377 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f6336ffd700 1 --2- 172.21.15.7:0/3635106377 >> v2:172.21.15.7:3300/0 conn(0x7f63380cc450 0x7f63380af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f6336ffd700 1 -- 172.21.15.7:0/3635106377 >> v1:172.21.15.50:6789/0 conn(0x7f63380ae7c0 legacy=0x7f6338155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f63367fc700 1 -- 172.21.15.7:0/3635106377 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602607464 0 0) 0x7f633806dca0 con 0x7f6338152ac0 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f6336ffd700 1 -- 172.21.15.7:0/3635106377 >> v1:172.21.15.7:6789/0 conn(0x7f6338152ac0 legacy=0x7f63380cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f6336ffd700 1 -- 172.21.15.7:0/3635106377 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380afbc0 con 0x7f63380cc450 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f6336ffd700 1 --2- 172.21.15.7:0/3635106377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f63380af5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f632c009ac0 tx=0x7f632c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ee6e1997572b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.511+0000 7f63367fc700 1 -- 172.21.15.7:0/3635106377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b6a0 con 0x7f63380cc450 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f63367fc700 1 -- 172.21.15.7:0/3635106377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b820 con 0x7f63380cc450 2023-10-06T11:22:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f63367fc700 1 -- 172.21.15.7:0/3635106377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bbe0 con 0x7f63380cc450 2023-10-06T11:22:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- 172.21.15.7:0/3635106377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 msgr2=0x7f63380af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 --2- 172.21.15.7:0/3635106377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f63380af5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f632c009ac0 tx=0x7f632c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 Processor -- start 2023-10-06T11:22:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 -- start start 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- 172.21.15.7:0/3635106377 shutdown_connections 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 --2- 172.21.15.7:0/3635106377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f63380af5e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- 172.21.15.7:0/3635106377 >> 172.21.15.7:0/3635106377 conn(0x7f6338066840 msgr2=0x7f6338066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- 172.21.15.7:0/3635106377 shutdown_connections 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe1dc154500 0x7fe1dc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc15e3e0 con 0x7fe1dc0cbab0 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0accb0 con 0x7fe1dc154500 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e37d6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc161e90 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- 172.21.15.7:0/3635106377 wait complete. 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43566/0 (socket says 172.21.15.7:43566) 2023-10-06T11:22:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 -- 172.21.15.7:0/151502581 learned_addr learned my addr 172.21.15.7:0/151502581 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e0d71700 1 --2- 172.21.15.7:0/151502581 >> v2:172.21.15.50:3300/0 conn(0x7fe1dc154500 0x7fe1dc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 Processor -- start 2023-10-06T11:22:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 -- 172.21.15.7:0/151502581 >> v2:172.21.15.50:3300/0 conn(0x7fe1dc154500 msgr2=0x7fe1dc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 --2- 172.21.15.7:0/151502581 >> v2:172.21.15.50:3300/0 conn(0x7fe1dc154500 0x7fe1dc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 -- 172.21.15.7:0/151502581 >> v1:172.21.15.50:6789/0 conn(0x7fe1dc0cbab0 legacy=0x7fe1dc153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7fe1e1d73700 1 -- 172.21.15.7:0/151502581 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc06ecb0 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e1d73700 1 --2- 172.21.15.7:0/151502581 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009f20 tx=0x7fe1d4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de034919db6aaf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.515+0000 7f633dc35700 1 -- start start 2023-10-06T11:22:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1d3fff700 1 -- 172.21.15.7:0/151502581 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400e070 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1d3fff700 1 -- 172.21.15.7:0/151502581 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d400b9d0 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1d3fff700 1 -- 172.21.15.7:0/151502581 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40092e0 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f633815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6338152ac0 0x7f6338149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633814bd50 con 0x7f63380cc450 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633814bfa0 con 0x7f63380ae7c0 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633814c280 con 0x7f6338152ac0 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57954/0 (socket says 172.21.15.7:57954) 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 -- 172.21.15.7:0/346737753 learned_addr learned my addr 172.21.15.7:0/346737753 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f6337fff700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6338152ac0 0x7f6338149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 -- 172.21.15.7:0/151502581 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 msgr2=0x7fe1dc161860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 --2- 172.21.15.7:0/151502581 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009f20 tx=0x7fe1d4009840 comp rx=0 tx=0).stop 2023-10-06T11:22:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f6336ffd700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f633815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 -- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6338152ac0 msgr2=0x7f6338149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6338152ac0 0x7f6338149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 -- 172.21.15.7:0/346737753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 msgr2=0x7f633815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f633815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 -- 172.21.15.7:0/346737753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009420 con 0x7f63380ae7c0 2023-10-06T11:22:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 -- 172.21.15.7:0/151502581 shutdown_connections 2023-10-06T11:22:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 --2- 172.21.15.7:0/151502581 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0c7af0 0x7fe1dc161860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 --2- 172.21.15.7:0/151502581 >> v2:172.21.15.50:3300/0 conn(0x7fe1dc154500 0x7fe1dc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 -- 172.21.15.7:0/151502581 >> 172.21.15.7:0/151502581 conn(0x7fe1dc066840 msgr2=0x7fe1dc066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 -- 172.21.15.7:0/151502581 shutdown_connections 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7fe1e37d6700 1 -- 172.21.15.7:0/151502581 wait complete. 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f6336ffd700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f633815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f63377fe700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f632800d6e0 tx=0x7f6328005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800f6a0 con 0x7f63380ae7c0 2023-10-06T11:22:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633814c560 con 0x7f63380ae7c0 2023-10-06T11:22:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633816a9b0 con 0x7f63380ae7c0 2023-10-06T11:22:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632800f840 con 0x7f63380ae7c0 2023-10-06T11:22:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.519+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800fc00 con 0x7f63380ae7c0 2023-10-06T11:22:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 Processor -- start 2023-10-06T11:22:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 -- start start 2023-10-06T11:22:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6328030a20 con 0x7f63380ae7c0 2023-10-06T11:22:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7f631f7fe700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 0x7f632406b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f632807cbb0 con 0x7f63380ae7c0 2023-10-06T11:22:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1dc0c7af0 0x7fe1dc16d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 0x7fe1dc0c0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c1240 con 0x7fe1dc154500 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c1520 con 0x7fe1dc0c7af0 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e37d6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c1800 con 0x7fe1dc0cbab0 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41794/0 (socket says 172.21.15.7:41794) 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 -- 172.21.15.7:0/3810234710 learned_addr learned my addr 172.21.15.7:0/3810234710 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e0d71700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 0x7fe1dc0c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1572700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1dc0c7af0 0x7fe1dc16d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 -- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 msgr2=0x7fe1dc0c0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 0x7fe1dc0c0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 -- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1dc0c7af0 msgr2=0x7fe1dc16d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1dc0c7af0 0x7fe1dc16d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 -- 172.21.15.7:0/3810234710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4009ae0 con 0x7fe1dc154500 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e0d71700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 0x7fe1dc0c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7f6336ffd700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 0x7f632406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7fe1e1d73700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b610 tx=0x7fe1d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.523+0000 7f6336ffd700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 0x7f632406b290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63380cc850 tx=0x7f632c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4008ad0 con 0x7fe1dc154500 2023-10-06T11:22:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1e37d6700 1 -- 172.21.15.7:0/3810234710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0c1ae0 con 0x7fe1dc154500 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1e37d6700 1 -- 172.21.15.7:0/3810234710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc16d080 con 0x7fe1dc154500 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d4008c30 con 0x7fe1dc154500 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4026460 con 0x7fe1dc154500 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe1d4003040 con 0x7fe1dc154500 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.527+0000 7fe1d17fa700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 0x7fe1c8062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.531+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63180052b0 con 0x7f63380ae7c0 2023-10-06T11:22:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.531+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe1d407f280 con 0x7fe1dc154500 2023-10-06T11:22:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.531+0000 7fe1e1572700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 0x7fe1c8062be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.531+0000 7fe1e37d6700 1 -- 172.21.15.7:0/3810234710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc04be00 con 0x7fe1dc154500 2023-10-06T11:22:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.531+0000 7fe1e1572700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 0x7fe1c8062be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cb200 tx=0x7fe1cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.539+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe1d404f4a0 con 0x7fe1dc154500 2023-10-06T11:22:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.539+0000 7f97bbfff700 1 Processor -- start 2023-10-06T11:22:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.539+0000 7f97bbfff700 1 -- start start 2023-10-06T11:22:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f632804ccb0 con 0x7f63380ae7c0 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97bc0caf00 0x7f97bc15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc0c9f70 con 0x7f97bc154620 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97bc0ca530 con 0x7f97bc0caf00 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15e4d0 con 0x7f97bc154250 2023-10-06T11:22:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43588/0 (socket says 172.21.15.7:43588) 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 -- 172.21.15.7:0/1755395055 learned_addr learned my addr 172.21.15.7:0/1755395055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 -- 172.21.15.7:0/1755395055 >> v2:172.21.15.7:3300/0 conn(0x7f97bc0caf00 msgr2=0x7f97bc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 --2- 172.21.15.7:0/1755395055 >> v2:172.21.15.7:3300/0 conn(0x7f97bc0caf00 0x7f97bc15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 -- 172.21.15.7:0/1755395055 >> v1:172.21.15.7:6789/0 conn(0x7f97bc154620 legacy=0x7f97bc069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 -- 172.21.15.7:0/1755395055 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc15e7b0 con 0x7f97bc154250 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97baffd700 1 --2- 172.21.15.7:0/1755395055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009f20 tx=0x7f97b4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba4db7b39f9bd56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97b9ffb700 1 -- 172.21.15.7:0/1755395055 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400e070 con 0x7f97bc154250 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97b9ffb700 1 -- 172.21.15.7:0/1755395055 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b400ba40 con 0x7f97bc154250 2023-10-06T11:22:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97b9ffb700 1 -- 172.21.15.7:0/1755395055 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40092e0 con 0x7f97bc154250 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- 172.21.15.7:0/1755395055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 msgr2=0x7f97bc0ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 --2- 172.21.15.7:0/1755395055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009f20 tx=0x7f97b4009840 comp rx=0 tx=0).stop 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- 172.21.15.7:0/1755395055 shutdown_connections 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 --2- 172.21.15.7:0/1755395055 >> v2:172.21.15.7:3300/0 conn(0x7f97bc0caf00 0x7f97bc15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 --2- 172.21.15.7:0/1755395055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc154250 0x7f97bc0ca9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.543+0000 7f97bbfff700 1 -- 172.21.15.7:0/1755395055 >> 172.21.15.7:0/1755395055 conn(0x7f97bc0cc190 msgr2=0x7f97bc0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.547+0000 7f97bbfff700 1 -- 172.21.15.7:0/1755395055 shutdown_connections 2023-10-06T11:22:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.547+0000 7f97bbfff700 1 -- 172.21.15.7:0/1755395055 wait complete. 2023-10-06T11:22:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 Processor -- start 2023-10-06T11:22:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 -- start start 2023-10-06T11:22:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 0x7f97bc174090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97ba7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41812/0 (socket says 172.21.15.7:41812) 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97ba7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:57984/0 (socket says 172.21.15.7:57984) 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97baffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 0x7f97bc174090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97baffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 0x7f97bc174090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43592/0 (socket says 172.21.15.7:43592) 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 -- 172.21.15.7:0/3916163683 learned_addr learned my addr 172.21.15.7:0/3916163683 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc177bf0 con 0x7f97bc154620 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc159530 con 0x7f97bc154250 2023-10-06T11:22:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc1745d0 con 0x7f97bc0caf00 2023-10-06T11:22:20.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 -- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 msgr2=0x7f97bc174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 0x7f97bc174090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 -- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 msgr2=0x7f97bc176c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97bb7fe700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b4009ae0 con 0x7f97bc154620 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.551+0000 7f97ba7fc700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97bb7fe700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00b4e0 tx=0x7f97ac009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00f5e0 con 0x7f97bc154620 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc177630 con 0x7f97bc154620 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc1591c0 con 0x7f97bc154620 2023-10-06T11:22:20.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00f760 con 0x7f97bc154620 2023-10-06T11:22:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00fb20 con 0x7f97bc154620 2023-10-06T11:22:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c0052b0 con 0x7f97bc154620 2023-10-06T11:22:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.555+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f97ac024430 con 0x7f97bc154620 2023-10-06T11:22:20.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.559+0000 7f97a2ffd700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 0x7f97a8062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.559+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f97ac07d1e0 con 0x7f97bc154620 2023-10-06T11:22:20.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.563+0000 7f97baffd700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 0x7f97a8062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.563+0000 7f97baffd700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 0x7f97a8062a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009ab0 tx=0x7f97b4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.563+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f97ac04d4e0 con 0x7f97bc154620 2023-10-06T11:22:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 Processor -- start 2023-10-06T11:22:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 -- start start 2023-10-06T11:22:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f627c0cc1f0 0x7f627c161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c06e860 con 0x7f627c15d390 2023-10-06T11:22:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c06dc50 con 0x7f627c15cfc0 2023-10-06T11:22:20.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f62832b5700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c06ecf0 con 0x7f627c0cc1f0 2023-10-06T11:22:20.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6280850700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f627c15d390 0x7f627c0cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54928/0 (socket says 172.21.15.7:54928) 2023-10-06T11:22:20.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6280850700 1 -- 172.21.15.7:0/2987688923 learned_addr learned my addr 172.21.15.7:0/2987688923 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:20.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6281852700 1 --2- 172.21.15.7:0/2987688923 >> v2:172.21.15.50:3300/0 conn(0x7f627c0cc1f0 0x7f627c161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517293918 0 0) 0x7f627c06e860 con 0x7f627c15d390 2023-10-06T11:22:20.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6268003570 con 0x7f627c15d390 2023-10-06T11:22:20.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f9f37938700 1 Processor -- start 2023-10-06T11:22:20.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f9f37938700 1 -- start start 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1406560092 0 0) 0x7f6268003570 con 0x7f627c15d390 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 >> v2:172.21.15.50:3300/0 conn(0x7f627c0cc1f0 msgr2=0x7f627c161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 --2- 172.21.15.7:0/2987688923 >> v2:172.21.15.50:3300/0 conn(0x7f627c0cc1f0 0x7f627c161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 >> v1:172.21.15.50:6789/0 conn(0x7f627c15cfc0 legacy=0x7f627c0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c06df30 con 0x7f627c15d390 2023-10-06T11:22:20.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f626c001470 con 0x7f627c15d390 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 >> v1:172.21.15.7:6789/0 conn(0x7f627c15d390 legacy=0x7f627c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f6268003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6268004810 0x7f6268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6268005210 0x7f626800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c06ecf0 con 0x7f6268003a60 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626800c130 con 0x7f6268004810 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626800c2f0 con 0x7f6268005210 2023-10-06T11:22:20.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6281051700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f6268003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.575+0000 7f6280850700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6268004810 0x7f6268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281852700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6268005210 0x7f626800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 -- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6268005210 msgr2=0x7f626800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6268005210 0x7f626800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 -- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6268004810 msgr2=0x7f6268004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6268004810 0x7f6268004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 -- 172.21.15.7:0/2987688923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268003570 con 0x7f6268003a60 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281051700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f6268003e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62740020a0 tx=0x7f62740016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbd9ddf149b1e72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62740019e0 con 0x7f6268003a60 2023-10-06T11:22:20.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6281852700 1 -- 172.21.15.7:0/2987688923 reap_dead start 2023-10-06T11:22:20.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400c8b0 con 0x7f6268003a60 2023-10-06T11:22:20.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f6273fff700 1 -- 172.21.15.7:0/2987688923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400cc50 con 0x7f6268003a60 2023-10-06T11:22:20.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f30155340 0x7f9f300aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f3006af50 0x7f9f300c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f3006af50 0x7f9f300c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f3006af50 0x7f9f300c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41834/0 (socket says 172.21.15.7:41834) 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 -- 172.21.15.7:0/1537439578 learned_addr learned my addr 172.21.15.7:0/1537439578 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f356d4700 1 --2- 172.21.15.7:0/1537439578 >> v2:172.21.15.50:3300/0 conn(0x7f9f30155340 0x7f9f300aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 msgr2=0x7f6268003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f6268003e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62740020a0 tx=0x7f62740016d0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f30159050 con 0x7f9f3006af50 2023-10-06T11:22:20.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f30159330 con 0x7f9f30155340 2023-10-06T11:22:20.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f30159610 con 0x7f9f300c6dd0 2023-10-06T11:22:20.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f35ed5700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- 172.21.15.7:0/2987688923 shutdown_connections 2023-10-06T11:22:20.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 --2- 172.21.15.7:0/2987688923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f6268003e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- 172.21.15.7:0/2987688923 >> 172.21.15.7:0/2987688923 conn(0x7f627c066840 msgr2=0x7f627c066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- 172.21.15.7:0/2987688923 shutdown_connections 2023-10-06T11:22:20.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- 172.21.15.7:0/2987688923 wait complete. 2023-10-06T11:22:20.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 -- 172.21.15.7:0/1537439578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 msgr2=0x7f9f300ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 -- 172.21.15.7:0/1537439578 >> v2:172.21.15.50:3300/0 conn(0x7f9f30155340 msgr2=0x7f9f300aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/1537439578 >> v2:172.21.15.50:3300/0 conn(0x7f9f30155340 0x7f9f300aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 -- 172.21.15.7:0/1537439578 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f301598f0 con 0x7f9f3006af50 2023-10-06T11:22:20.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300c8440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f300c90c0 tx=0x7f9f28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f6c5f70ca3fb1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7fa963663700 1 Processor -- start 2023-10-06T11:22:20.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 Processor -- start 2023-10-06T11:22:20.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7f62832b5700 1 -- start start 2023-10-06T11:22:20.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.579+0000 7fa963663700 1 -- start start 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa95c0ca850 0x7fa95c156510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c158860 con 0x7fa95c14be20 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c156a50 con 0x7fa95c068940 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c06a360 con 0x7fa95c0ca850 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa95c0ca850 0x7fa95c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa95c0ca850 0x7fa95c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41840/0 (socket says 172.21.15.7:41840) 2023-10-06T11:22:20.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 -- 172.21.15.7:0/2606831951 learned_addr learned my addr 172.21.15.7:0/2606831951 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 -- 172.21.15.7:0/2606831951 >> v1:172.21.15.50:6789/0 conn(0x7fa95c068940 legacy=0x7fa95c158a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:22:20.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 -- 172.21.15.7:0/2606831951 >> v1:172.21.15.7:6789/0 conn(0x7fa95c14be20 legacy=0x7fa95c0ca140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:22:20.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 -- 172.21.15.7:0/2606831951 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c069d50 con 0x7fa95c0ca850 2023-10-06T11:22:20.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa961c00700 1 --2- 172.21.15.7:0/2606831951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c156510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06d5f0 tx=0x7fa958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e3c052f5415e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 Processor -- start 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 -- start start 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f627c147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c148b80 0x7f627c14f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c148ff0 con 0x7f6268003a60 2023-10-06T11:22:20.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c14f020 con 0x7f627c148b80 2023-10-06T11:22:20.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c14ea10 con 0x7f627c14d070 2023-10-06T11:22:20.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f9f27fff700 1 -- 172.21.15.7:0/1537439578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280092e0 con 0x7f9f3006af50 2023-10-06T11:22:20.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f9f27fff700 1 -- 172.21.15.7:0/1537439578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2800b970 con 0x7f9f3006af50 2023-10-06T11:22:20.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f9f27fff700 1 -- 172.21.15.7:0/1537439578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800bd30 con 0x7f9f3006af50 2023-10-06T11:22:20.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6281852700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c148b80 0x7f627c14f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43612/0 (socket says 172.21.15.7:43612) 2023-10-06T11:22:20.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 -- 172.21.15.7:0/3343730282 learned_addr learned my addr 172.21.15.7:0/3343730282 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3ab8160570 0x7f3ab80aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ab8160d80 0x7f3ab80af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80ac0e0 con 0x7f3ab8160940 2023-10-06T11:22:20.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80ab4d0 con 0x7f3ab8160d80 2023-10-06T11:22:20.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abe682700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80ab7b0 con 0x7f3ab8160570 2023-10-06T11:22:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6281051700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f627c147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa953fff700 1 -- 172.21.15.7:0/2606831951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800b7e0 con 0x7fa95c0ca850 2023-10-06T11:22:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa953fff700 1 -- 172.21.15.7:0/2606831951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800b940 con 0x7fa95c0ca850 2023-10-06T11:22:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa953fff700 1 -- 172.21.15.7:0/2606831951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800bce0 con 0x7fa95c0ca850 2023-10-06T11:22:20.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ab8160d80 0x7f3ab80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abce7f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ab8160940 0x7f3ab815d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54930/0 (socket says 172.21.15.7:54930) 2023-10-06T11:22:20.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abce7f700 1 -- 172.21.15.7:0/251341962 learned_addr learned my addr 172.21.15.7:0/251341962 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 -- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c148b80 msgr2=0x7f627c14f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c148b80 0x7f627c14f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 -- 172.21.15.7:0/3343730282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 msgr2=0x7f627c147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f627c147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 -- 172.21.15.7:0/3343730282 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c150760 con 0x7f627c14d070 2023-10-06T11:22:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abd680700 1 --2- 172.21.15.7:0/251341962 >> v2:172.21.15.50:3300/0 conn(0x7f3ab8160570 0x7f3ab80aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837488116 0 0) 0x7f3ab80ac0e0 con 0x7f3ab8160940 2023-10-06T11:22:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa4003570 con 0x7f3ab8160940 2023-10-06T11:22:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 -- 172.21.15.7:0/251341962 >> v2:172.21.15.50:3300/0 conn(0x7f3ab8160570 msgr2=0x7f3ab80aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 --2- 172.21.15.7:0/251341962 >> v2:172.21.15.50:3300/0 conn(0x7f3ab8160570 0x7f3ab80aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f6280850700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f626c0073c0 tx=0x7f627c06e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266918280 0 0) 0x7f3aa4003570 con 0x7f3ab8160940 2023-10-06T11:22:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 -- 172.21.15.7:0/251341962 >> v1:172.21.15.7:6789/0 conn(0x7f3ab8160940 legacy=0x7f3ab815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 -- 172.21.15.7:0/251341962 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80aba90 con 0x7f3ab8160d80 2023-10-06T11:22:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c007780 con 0x7f627c14d070 2023-10-06T11:22:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c00f900 con 0x7f627c14d070 2023-10-06T11:22:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00fcc0 con 0x7f627c14d070 2023-10-06T11:22:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3abde81700 1 --2- 172.21.15.7:0/251341962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160d80 0x7f3ab80af240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80ae470 tx=0x7f3ab40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981d74470b065919 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00017f0 con 0x7f3ab8160d80 2023-10-06T11:22:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 msgr2=0x7f9f300c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f9f37938700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300c8440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f300c90c0 tx=0x7f9f28009840 comp rx=0 tx=0).stop 2023-10-06T11:22:20.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab4011420 con 0x7f3ab8160d80 2023-10-06T11:22:20.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f3aaffff700 1 -- 172.21.15.7:0/251341962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40117a0 con 0x7f3ab8160d80 2023-10-06T11:22:20.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c14b890 con 0x7f627c14d070 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c14ecb0 con 0x7f627c14d070 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- 172.21.15.7:0/2606831951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 msgr2=0x7fa95c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 --2- 172.21.15.7:0/2606831951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c156510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06d5f0 tx=0x7fa958009250 comp rx=0 tx=0).stop 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- 172.21.15.7:0/2606831951 shutdown_connections 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 --2- 172.21.15.7:0/2606831951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c156510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- 172.21.15.7:0/2606831951 >> 172.21.15.7:0/2606831951 conn(0x7fa95c067e30 msgr2=0x7fa95c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.583+0000 7fa963663700 1 -- 172.21.15.7:0/2606831951 shutdown_connections 2023-10-06T11:22:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 -- 172.21.15.7:0/2606831951 wait complete. 2023-10-06T11:22:20.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f3abe682700 1 -- 172.21.15.7:0/251341962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160d80 msgr2=0x7f3ab80af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f3abe682700 1 --2- 172.21.15.7:0/251341962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160d80 0x7f3ab80af240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80ae470 tx=0x7f3ab40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 Processor -- start 2023-10-06T11:22:20.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 shutdown_connections 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 --2- 172.21.15.7:0/1537439578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300c8440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 --2- 172.21.15.7:0/1537439578 >> v2:172.21.15.50:3300/0 conn(0x7f9f30155340 0x7f9f300aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 >> 172.21.15.7:0/1537439578 conn(0x7f9f30067e30 msgr2=0x7f9f30068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 -- start start 2023-10-06T11:22:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f626c01d460 con 0x7f627c14d070 2023-10-06T11:22:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 shutdown_connections 2023-10-06T11:22:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 -- 172.21.15.7:0/1537439578 wait complete. 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa95c068940 0x7fa95c14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa95c14be20 0x7fa95c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c14cbc0 con 0x7fa95c0ca850 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c153c00 con 0x7fa95c14be20 2023-10-06T11:22:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa963663700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c154720 con 0x7fa95c068940 2023-10-06T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41848/0 (socket says 172.21.15.7:41848) 2023-10-06T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 -- 172.21.15.7:0/1579349366 learned_addr learned my addr 172.21.15.7:0/1579349366 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f62717fa700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 0x7f6260062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f626c07ce10 con 0x7f627c14d070 2023-10-06T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f6281051700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 0x7f6260062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f6281051700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 0x7f6260062a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6274001e30 tx=0x7f6274016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 Processor -- start 2023-10-06T11:22:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa9613ff700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa95c068940 0x7fa95c14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa961c00700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa95c14be20 0x7fa95c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 -- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa95c068940 msgr2=0x7fa95c14ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa95c068940 0x7fa95c14ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 -- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa95c14be20 msgr2=0x7fa95c14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa95c14be20 0x7fa95c14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 -- 172.21.15.7:0/1579349366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580094f0 con 0x7fa95c0ca850 2023-10-06T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7fa960bfe700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b5e0 tx=0x7fa94c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.587+0000 7f9f37938700 1 -- start start 2023-10-06T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- 172.21.15.7:0/251341962 shutdown_connections 2023-10-06T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 --2- 172.21.15.7:0/251341962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160d80 0x7f3ab80af240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 --2- 172.21.15.7:0/251341962 >> v2:172.21.15.50:3300/0 conn(0x7f3ab8160570 0x7f3ab80aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- 172.21.15.7:0/251341962 >> 172.21.15.7:0/251341962 conn(0x7f3ab8068900 msgr2=0x7f3ab80cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- 172.21.15.7:0/251341962 shutdown_connections 2023-10-06T11:22:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- 172.21.15.7:0/251341962 wait complete. 2023-10-06T11:22:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f30155340 0x7f9f300c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b1800 con 0x7f9f3006af50 2023-10-06T11:22:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b00c0 con 0x7f9f30155340 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f37938700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300adc10 con 0x7f9f300c6dd0 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41854/0 (socket says 172.21.15.7:41854) 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 -- 172.21.15.7:0/4154265450 learned_addr learned my addr 172.21.15.7:0/4154265450 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 -- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 msgr2=0x7f9f300c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 -- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f30155340 msgr2=0x7f9f300c7770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:22:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f30155340 0x7f9f300c7770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 -- 172.21.15.7:0/4154265450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009ae0 con 0x7f9f3006af50 2023-10-06T11:22:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f9f356d4700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000fe50 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 Processor -- start 2023-10-06T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- start start 2023-10-06T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 0x7f3ab80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ab8160d80 0x7f3ab816d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80cb580 con 0x7f3ab8160940 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80c9ec0 con 0x7f3ab8160570 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abe682700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80c92b0 con 0x7f3ab8160d80 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41858/0 (socket says 172.21.15.7:41858) 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 -- 172.21.15.7:0/3531558045 learned_addr learned my addr 172.21.15.7:0/3531558045 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abd680700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 0x7f3ab80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abde81700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ab8160d80 0x7f3ab816d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 -- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ab8160d80 msgr2=0x7f3ab816d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ab8160d80 0x7f3ab816d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 -- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 msgr2=0x7f3ab80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 0x7f3ab80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 -- 172.21.15.7:0/3531558045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4009420 con 0x7f3ab8160940 2023-10-06T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abd680700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 0x7f3ab80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.591+0000 7f3abce7f700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cc10 tx=0x7f3ab0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000c070 con 0x7f9f3006af50 2023-10-06T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000dbf0 con 0x7f9f3006af50 2023-10-06T11:22:20.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3016e190 con 0x7f9f3006af50 2023-10-06T11:22:20.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2001a480 con 0x7f9f3006af50 2023-10-06T11:22:20.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c04be00 con 0x7f627c14d070 2023-10-06T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300afa70 con 0x7f9f3006af50 2023-10-06T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f100052b0 con 0x7f9f3006af50 2023-10-06T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9f2001a830 con 0x7f9f3006af50 2023-10-06T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 0x7f9f14062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9f20015070 con 0x7f9f3006af50 2023-10-06T11:22:20.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 0x7f9f14062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.595+0000 7f9f34ed3700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 0x7f9f14062b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009fd0 tx=0x7f9f2801a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00f510 con 0x7fa95c0ca850 2023-10-06T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c0c88e0 con 0x7fa95c0ca850 2023-10-06T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c00f690 con 0x7fa95c0ca850 2023-10-06T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f5cca38a700 1 Processor -- start 2023-10-06T11:22:20.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f2004d540 con 0x7f9f3006af50 2023-10-06T11:22:20.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00fa50 con 0x7fa95c0ca850 2023-10-06T11:22:20.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1540d0 con 0x7fa95c0ca850 2023-10-06T11:22:20.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f5cca38a700 1 -- start start 2023-10-06T11:22:20.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004dc0 con 0x7f3ab8160940 2023-10-06T11:22:20.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab000a6b0 con 0x7f3ab8160940 2023-10-06T11:22:20.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.599+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000aa70 con 0x7f3ab8160940 2023-10-06T11:22:20.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80c9590 con 0x7f3ab8160940 2023-10-06T11:22:20.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80c9b50 con 0x7f3ab8160940 2023-10-06T11:22:20.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cca38a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5cc40699d0 0x7f5cc415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cca38a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 0x7f5cc40c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cca38a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc40c6d60 con 0x7f5cc4156ad0 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cca38a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc406aa70 con 0x7f5cc40699d0 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cca38a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4069e60 con 0x7f5cc4156ea0 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc8927700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 0x7f5cc40c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5cc40699d0 0x7f5cc415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc8927700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 0x7f5cc40c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:43656/0 (socket says 172.21.15.7:43656) 2023-10-06T11:22:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc8927700 1 -- 172.21.15.7:0/1280194343 learned_addr learned my addr 172.21.15.7:0/1280194343 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c070f20 con 0x7fa95c0ca850 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 -- 172.21.15.7:0/1280194343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 msgr2=0x7f5cc40c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 --2- 172.21.15.7:0/1280194343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 0x7f5cc40c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 -- 172.21.15.7:0/1280194343 >> v1:172.21.15.50:6789/0 conn(0x7f5cc4156ad0 legacy=0x7f5cc406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 -- 172.21.15.7:0/1280194343 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc406a140 con 0x7f5cc40699d0 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc37fe700 1 --2- 172.21.15.7:0/1280194343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40699d0 0x7f5cc415b2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098d0 tx=0x7f5cb8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763c5f2f922a93c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc2ffd700 1 -- 172.21.15.7:0/1280194343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8005c10 con 0x7f5cc40699d0 2023-10-06T11:22:20.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc2ffd700 1 -- 172.21.15.7:0/1280194343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8005d70 con 0x7f5cc40699d0 2023-10-06T11:22:20.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f5cc2ffd700 1 -- 172.21.15.7:0/1280194343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80108e0 con 0x7f5cc40699d0 2023-10-06T11:22:20.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa94c01a400 con 0x7fa95c0ca850 2023-10-06T11:22:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3ab0007150 con 0x7f3ab8160940 2023-10-06T11:22:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f626c04cfb0 con 0x7f627c14d070 2023-10-06T11:22:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3aad7fa700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 0x7f3a9c062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3ab007c7b0 con 0x7f3ab8160940 2023-10-06T11:22:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.603+0000 7f3abd680700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 0x7f3a9c062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7fa9517fa700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 0x7fa940062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa94c07c510 con 0x7fa95c0ca850 2023-10-06T11:22:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7fa9613ff700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 0x7fa940062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab806c710 con 0x7f3ab8160940 2023-10-06T11:22:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f3abd680700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 0x7f3a9c062aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80aac30 tx=0x7f3aa8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 -- 172.21.15.7:0/1280194343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40699d0 msgr2=0x7f5cc415b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 --2- 172.21.15.7:0/1280194343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40699d0 0x7f5cc415b2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098d0 tx=0x7f5cb8009120 comp rx=0 tx=0).stop 2023-10-06T11:22:20.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7fa9613ff700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 0x7fa940062b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa954007580 tx=0x7fa954009760 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 -- 172.21.15.7:0/1280194343 shutdown_connections 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 --2- 172.21.15.7:0/1280194343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc4156ea0 0x7f5cc40c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 --2- 172.21.15.7:0/1280194343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40699d0 0x7f5cc415b2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 -- 172.21.15.7:0/1280194343 >> 172.21.15.7:0/1280194343 conn(0x7f5cc4067e30 msgr2=0x7f5cc4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 -- 172.21.15.7:0/1280194343 shutdown_connections 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 -- 172.21.15.7:0/1280194343 wait complete. 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.607+0000 7f5cca38a700 1 Processor -- start 2023-10-06T11:22:20.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 -- start start 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc40699d0 0x7f5cc4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4168910 con 0x7f5cc4156ea0 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4165880 con 0x7f5cc4156ad0 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cca38a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4165b60 con 0x7f5cc40699d0 2023-10-06T11:22:20.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:58044/0 (socket says 172.21.15.7:58044) 2023-10-06T11:22:20.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc37fe700 1 -- 172.21.15.7:0/3180728125 learned_addr learned my addr 172.21.15.7:0/3180728125 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:20.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41880/0 (socket says 172.21.15.7:41880) 2023-10-06T11:22:20.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc3fff700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc40699d0 0x7f5cc4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 -- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc40699d0 msgr2=0x7f5cc4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc40699d0 0x7f5cc4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 -- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 msgr2=0x7f5cc40c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 -- 172.21.15.7:0/3180728125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80093c0 con 0x7f5cc4156ea0 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc37fe700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:20.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.611+0000 7f5cc8927700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00ec90 tx=0x7f5cbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.615+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa94c04c710 con 0x7fa95c0ca850 2023-10-06T11:22:20.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.623+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3ab004cab0 con 0x7f3ab8160940 2023-10-06T11:22:20.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.627+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00c680 con 0x7f5cc4156ea0 2023-10-06T11:22:20.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.627+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc00f040 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.627+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00ca30 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc4165e40 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc40c33f0 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5cbc014440 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cabfff700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 0x7f5cb0062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5cbc07d040 con 0x7f5cc4156ea0 2023-10-06T11:22:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cc3fff700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 0x7f5cb0062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:20.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.635+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4070f20 con 0x7f5cc4156ea0 2023-10-06T11:22:20.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.639+0000 7f5cc3fff700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 0x7f5cb0062ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4006fd0 tx=0x7f5cb40090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:20.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.647+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5cbc04d260 con 0x7f5cc4156ea0 2023-10-06T11:22:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.831+0000 7fe1e37d6700 1 -- 172.21.15.7:0/3810234710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe1dc0c8fd0 con 0x7fe1dc154500 2023-10-06T11:22:20.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.831+0000 7fe1d17fa700 1 -- 172.21.15.7:0/3810234710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1d4021020 con 0x7fe1dc154500 2023-10-06T11:22:20.837 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 msgr2=0x7fe1c8062be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 0x7fe1c8062be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cb200 tx=0x7fe1cc009040 comp rx=0 tx=0).stop 2023-10-06T11:22:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 msgr2=0x7fe1dc0afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b610 tx=0x7fe1d4018000 comp rx=0 tx=0).stop 2023-10-06T11:22:20.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 shutdown_connections 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe1c8060730 0x7fe1c8062be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc154500 0x7fe1dc0afa20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1dc0cbab0 0x7fe1dc0c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 --2- 172.21.15.7:0/3810234710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1dc0c7af0 0x7fe1dc16d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 >> 172.21.15.7:0/3810234710 conn(0x7fe1dc066840 msgr2=0x7fe1dc0ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 shutdown_connections 2023-10-06T11:22:20.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.843+0000 7fe1c6ffd700 1 -- 172.21.15.7:0/3810234710 wait complete. 2023-10-06T11:22:20.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.855+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6318005560 con 0x7f63380ae7c0 2023-10-06T11:22:20.860 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:20.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.855+0000 7f631f7fe700 1 -- 172.21.15.7:0/346737753 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6328014030 con 0x7f63380ae7c0 2023-10-06T11:22:20.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-06T11:22:20.862 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 msgr2=0x7f632406b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 0x7f632406b290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63380cc850 tx=0x7f632c016000 comp rx=0 tx=0).stop 2023-10-06T11:22:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 msgr2=0x7f633816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f632800d6e0 tx=0x7f6328005ea0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 shutdown_connections 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6324068de0 0x7f632406b290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6338152ac0 0x7f6338149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63380cc450 0x7f633815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 --2- 172.21.15.7:0/346737753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63380ae7c0 0x7f633816a470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 >> 172.21.15.7:0/346737753 conn(0x7f6338066840 msgr2=0x7f6338153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 shutdown_connections 2023-10-06T11:22:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.859+0000 7f633dc35700 1 -- 172.21.15.7:0/346737753 wait complete. 2023-10-06T11:22:20.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-06T11:22:20.877 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f979c005830 con 0x7f97bc154620 2023-10-06T11:22:20.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97a2ffd700 1 -- 172.21.15.7:0/3916163683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97ac056030 con 0x7f97bc154620 2023-10-06T11:22:20.879 INFO:teuthology.orchestra.run.smithi007.stdout:55834574856 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 msgr2=0x7f97a8062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 0x7f97a8062a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009ab0 tx=0x7f97b4017000 comp rx=0 tx=0).stop 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 msgr2=0x7f97bc0b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00b4e0 tx=0x7f97ac009060 comp rx=0 tx=0).stop 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 shutdown_connections 2023-10-06T11:22:20.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f97a80605d0 0x7f97a8062a80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97bc154620 0x7f97bc0b13c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97bc154250 0x7f97bc176c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 --2- 172.21.15.7:0/3916163683 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f97bc0caf00 0x7f97bc174090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 >> 172.21.15.7:0/3916163683 conn(0x7f97bc0cc190 msgr2=0x7f97bc0c2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 shutdown_connections 2023-10-06T11:22:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.875+0000 7f97bbfff700 1 -- 172.21.15.7:0/3916163683 wait complete. 2023-10-06T11:22:20.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-06T11:22:20.891 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.891+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ab80ca000 con 0x7f3ab8160940 2023-10-06T11:22:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.891+0000 7f3aad7fa700 1 -- 172.21.15.7:0/3531558045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ab000ae60 con 0x7f3ab8160940 2023-10-06T11:22:20.897 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 msgr2=0x7f3a9c062aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 0x7f3a9c062aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80aac30 tx=0x7f3aa8008040 comp rx=0 tx=0).stop 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 msgr2=0x7f3ab815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cc10 tx=0x7f3ab0001640 comp rx=0 tx=0).stop 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 shutdown_connections 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3a9c0605f0 0x7f3a9c062aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ab8160d80 0x7f3ab816d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ab8160940 0x7f3ab815ad40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 --2- 172.21.15.7:0/3531558045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ab8160570 0x7f3ab80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 >> 172.21.15.7:0/3531558045 conn(0x7f3ab8068900 msgr2=0x7f3ab80695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 shutdown_connections 2023-10-06T11:22:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.895+0000 7f3abe682700 1 -- 172.21.15.7:0/3531558045 wait complete. 2023-10-06T11:22:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.899+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f627c0c8fb0 con 0x7f627c14d070 2023-10-06T11:22:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.899+0000 7f62717fa700 1 -- 172.21.15.7:0/3343730282 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f626c0505d0 con 0x7f627c14d070 2023-10-06T11:22:20.904 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2023-10-06T11:22:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 msgr2=0x7f6260062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 0x7f6260062a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6274001e30 tx=0x7f6274016000 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 msgr2=0x7f627c148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f626c0073c0 tx=0x7f627c06e7d0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 shutdown_connections 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6260060550 0x7f6260062a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c148b80 0x7f627c14f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f627c14d070 0x7f627c148640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 --2- 172.21.15.7:0/3343730282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6268003a60 0x7f627c147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 >> 172.21.15.7:0/3343730282 conn(0x7f627c066840 msgr2=0x7f627c0cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-06T11:22:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 shutdown_connections 2023-10-06T11:22:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.903+0000 7f62832b5700 1 -- 172.21.15.7:0/3343730282 wait complete. 2023-10-06T11:22:20.910 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.911+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9f10005830 con 0x7f9f3006af50 2023-10-06T11:22:20.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.911+0000 7f9f257fa700 1 -- 172.21.15.7:0/4154265450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f20050b60 con 0x7f9f3006af50 2023-10-06T11:22:20.916 INFO:teuthology.orchestra.run.smithi007.stdout:55834574857 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 msgr2=0x7f9f14062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 0x7f9f14062b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009fd0 tx=0x7f9f2801a000 comp rx=0 tx=0).stop 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 msgr2=0x7f9f300abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000fe50 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 shutdown_connections 2023-10-06T11:22:20.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060650 0x7f9f14062b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f30155340 0x7f9f300c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f300c6dd0 0x7f9f300c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 --2- 172.21.15.7:0/4154265450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f3006af50 0x7f9f300abd10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 >> 172.21.15.7:0/4154265450 conn(0x7f9f30067e30 msgr2=0x7f9f3014f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 shutdown_connections 2023-10-06T11:22:20.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.915+0000 7f9f37938700 1 -- 172.21.15.7:0/4154265450 wait complete. 2023-10-06T11:22:20.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-10-06T11:22:20.921 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa95c14c2a0 con 0x7fa95c0ca850 2023-10-06T11:22:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa9517fa700 1 -- 172.21.15.7:0/1579349366 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa94c04fd30 con 0x7fa95c0ca850 2023-10-06T11:22:20.926 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 msgr2=0x7fa940062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 0x7fa940062b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa954007580 tx=0x7fa954009760 comp rx=0 tx=0).stop 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 msgr2=0x7fa95c13e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b5e0 tx=0x7fa94c0090f0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 shutdown_connections 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9400606a0 0x7fa940062b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa95c14be20 0x7fa95c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95c0ca850 0x7fa95c13e930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 --2- 172.21.15.7:0/1579349366 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa95c068940 0x7fa95c14ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 >> 172.21.15.7:0/1579349366 conn(0x7fa95c067e30 msgr2=0x7fa95c155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 shutdown_connections 2023-10-06T11:22:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.919+0000 7fa963663700 1 -- 172.21.15.7:0/1579349366 wait complete. 2023-10-06T11:22:20.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-10-06T11:22:20.931 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-10-06T11:22:20.935 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.935+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5cc406a420 con 0x7f5cc4156ea0 2023-10-06T11:22:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.935+0000 7f5cabfff700 1 -- 172.21.15.7:0/3180728125 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cbc050880 con 0x7f5cc4156ea0 2023-10-06T11:22:20.941 INFO:teuthology.orchestra.run.smithi007.stdout:55834574855 2023-10-06T11:22:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 msgr2=0x7f5cb0062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 0x7f5cb0062ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4006fd0 tx=0x7f5cb40090f0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 msgr2=0x7f5cc40c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00ec90 tx=0x7f5cbc00b040 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 shutdown_connections 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5cb0060620 0x7f5cb0062ad0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc4156ea0 0x7f5cc40c9af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc4156ad0 0x7f5cc40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 --2- 172.21.15.7:0/3180728125 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5cc40699d0 0x7f5cc4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 >> 172.21.15.7:0/3180728125 conn(0x7f5cc4067e30 msgr2=0x7f5cc4146180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 shutdown_connections 2023-10-06T11:22:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:20.939+0000 7f5cca38a700 1 -- 172.21.15.7:0/3180728125 wait complete. 2023-10-06T11:22:20.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T11:22:20.953 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:20.953 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T11:22:20.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:22:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 Processor -- start 2023-10-06T11:22:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 -- start start 2023-10-06T11:22:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05e00b1150 0x7f05e0156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00cabe0 con 0x7f05e0158660 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00caec0 con 0x7f05e01569d0 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.047+0000 7f05e4c0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00cb1a0 con 0x7f05e00b1150 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05deffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05e0158660 0x7f05e00b0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54944/0 (socket says 172.21.15.7:54944) 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05deffd700 1 -- 172.21.15.7:0/3494872836 learned_addr learned my addr 172.21.15.7:0/3494872836 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05dffff700 1 --2- 172.21.15.7:0/3494872836 >> v2:172.21.15.7:3300/0 conn(0x7f05e00b1150 0x7f05e0156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05dffff700 1 -- 172.21.15.7:0/3494872836 >> v1:172.21.15.50:6789/0 conn(0x7f05e01569d0 legacy=0x7f05e00c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05dffff700 1 -- 172.21.15.7:0/3494872836 >> v1:172.21.15.7:6789/0 conn(0x7f05e0158660 legacy=0x7f05e00b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05dffff700 1 -- 172.21.15.7:0/3494872836 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00c8ad0 con 0x7f05e00b1150 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05dffff700 1 --2- 172.21.15.7:0/3494872836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e00b1150 0x7f05e0156da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5400dd94b9da84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05de7fc700 1 -- 172.21.15.7:0/3494872836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8009e80 con 0x7f05e00b1150 2023-10-06T11:22:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05de7fc700 1 -- 172.21.15.7:0/3494872836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800b820 con 0x7f05e00b1150 2023-10-06T11:22:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05de7fc700 1 -- 172.21.15.7:0/3494872836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800bba0 con 0x7f05e00b1150 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3494872836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e00b1150 msgr2=0x7f05e0156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3494872836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e00b1150 0x7f05e0156da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3494872836 shutdown_connections 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3494872836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e00b1150 0x7f05e0156da0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3494872836 >> 172.21.15.7:0/3494872836 conn(0x7f05e00688a0 msgr2=0x7f05e00cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3494872836 shutdown_connections 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3494872836 wait complete. 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 Processor -- start 2023-10-06T11:22:21.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- start start 2023-10-06T11:22:21.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e00b1150 0x7f05e0149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f05e0158660 0x7f05e0153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015d460 con 0x7f05e01569d0 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b07c0 con 0x7f05e00b1150 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05e4c0c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00afbb0 con 0x7f05e0158660 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41896/0 (socket says 172.21.15.7:41896) 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.051+0000 7f05deffd700 1 -- 172.21.15.7:0/3742672282 learned_addr learned my addr 172.21.15.7:0/3742672282 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05df7fe700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e00b1150 0x7f05e0149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05dffff700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f05e0158660 0x7f05e0153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 -- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f05e0158660 msgr2=0x7f05e0153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f05e0158660 0x7f05e0153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 -- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e00b1150 msgr2=0x7f05e0149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e00b1150 0x7f05e0149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 -- 172.21.15.7:0/3742672282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8009420 con 0x7f05e01569d0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05deffd700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c560 tx=0x7f05c8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8005af0 con 0x7f05e01569d0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8005c70 con 0x7f05e01569d0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e00afe90 con 0x7f05e01569d0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80070d0 con 0x7f05e01569d0 2023-10-06T11:22:21.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00b04d0 con 0x7f05e01569d0 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e006c710 con 0x7f05e01569d0 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.055+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f05c80074c0 con 0x7f05e01569d0 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.059+0000 7f05d77fe700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 0x7f05c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.059+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f05c807bfe0 con 0x7f05e01569d0 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.059+0000 7f05df7fe700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 0x7f05c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.059+0000 7f05df7fe700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 0x7f05c4062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007fd0 tx=0x7f05d00026a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.067+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f05c804c0e0 con 0x7f05e01569d0 2023-10-06T11:22:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.211+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05e00cc550 con 0x7f05c4060550 2023-10-06T11:22:21.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.231+0000 7f05d77fe700 1 -- 172.21.15.7:0/3742672282 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367520 (secure 0 0 0) 0x7f05e00cc550 con 0x7f05c4060550 2023-10-06T11:22:21.241 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.235+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 msgr2=0x7f05c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.235+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 0x7f05c4062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007fd0 tx=0x7f05d00026a0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.235+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 msgr2=0x7f05e0150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.235+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c560 tx=0x7f05c8007f60 comp rx=0 tx=0).stop 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 shutdown_connections 2023-10-06T11:22:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f05c4060550 0x7f05c4062a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f05e0158660 0x7f05e0153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05e01569d0 0x7f05e0150600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 --2- 172.21.15.7:0/3742672282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05e00b1150 0x7f05e0149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 >> 172.21.15.7:0/3742672282 conn(0x7f05e00688a0 msgr2=0x7f05e0154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 shutdown_connections 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.239+0000 7f05e4c0c700 1 -- 172.21.15.7:0/3742672282 wait complete. 2023-10-06T11:22:21.245 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:22:21.260 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-06T11:22:19.818682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414376,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752560344,"statfs":{"total":773094113280,"available":770621792256,"internally_reserved":0,"allocated":3383296,"data_stored":2065368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004789"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642766+0000","last_change":"2023-10-06T11:22:01.634221+0000","last_active":"2023-10-06T11:22:01.642766+0000","last_peered":"2023-10-06T11:22:01.642766+0000","last_clean":"2023-10-06T11:22:01.642766+0000","last_became_active":"2023-10-06T11:22:01.633727+0000","last_became_peered":"2023-10-06T11:22:01.633727+0000","last_unstale":"2023-10-06T11:22:01.642766+0000","last_undegraded":"2023-10-06T11:22:01.642766+0000","last_fullsized":"2023-10-06T11:22:01.642766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627705+0000","last_change":"2023-10-06T11:21:56.568541+0000","last_active":"2023-10-06T11:22:01.627705+0000","last_peered":"2023-10-06T11:22:01.627705+0000","last_clean":"2023-10-06T11:22:01.627705+0000","last_became_active":"2023-10-06T11:21:56.567104+0000","last_became_peered":"2023-10-06T11:21:56.567104+0000","last_unstale":"2023-10-06T11:22:01.627705+0000","last_undegraded":"2023-10-06T11:22:01.627705+0000","last_fullsized":"2023-10-06T11:22:01.627705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612705+0000","last_change":"2023-10-06T11:21:57.566064+0000","last_active":"2023-10-06T11:22:01.612705+0000","last_peered":"2023-10-06T11:22:01.612705+0000","last_clean":"2023-10-06T11:22:01.612705+0000","last_became_active":"2023-10-06T11:21:57.565589+0000","last_became_peered":"2023-10-06T11:21:57.565589+0000","last_unstale":"2023-10-06T11:22:01.612705+0000","last_undegraded":"2023-10-06T11:22:01.612705+0000","last_fullsized":"2023-10-06T11:22:01.612705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645194+0000","last_change":"2023-10-06T11:22:01.637184+0000","last_active":"2023-10-06T11:22:01.645194+0000","last_peered":"2023-10-06T11:22:01.645194+0000","last_clean":"2023-10-06T11:22:01.645194+0000","last_became_active":"2023-10-06T11:22:01.636773+0000","last_became_peered":"2023-10-06T11:22:01.636773+0000","last_unstale":"2023-10-06T11:22:01.645194+0000","last_undegraded":"2023-10-06T11:22:01.645194+0000","last_fullsized":"2023-10-06T11:22:01.645194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:41.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617318+0000","last_change":"2023-10-06T11:21:56.565259+0000","last_active":"2023-10-06T11:22:01.617318+0000","last_peered":"2023-10-06T11:22:01.617318+0000","last_clean":"2023-10-06T11:22:01.617318+0000","last_became_active":"2023-10-06T11:21:56.564927+0000","last_became_peered":"2023-10-06T11:21:56.564927+0000","last_unstale":"2023-10-06T11:22:01.617318+0000","last_undegraded":"2023-10-06T11:22:01.617318+0000","last_fullsized":"2023-10-06T11:22:01.617318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617577+0000","last_change":"2023-10-06T11:21:57.562278+0000","last_active":"2023-10-06T11:22:01.617577+0000","last_peered":"2023-10-06T11:22:01.617577+0000","last_clean":"2023-10-06T11:22:01.617577+0000","last_became_active":"2023-10-06T11:21:57.561972+0000","last_became_peered":"2023-10-06T11:21:57.561972+0000","last_unstale":"2023-10-06T11:22:01.617577+0000","last_undegraded":"2023-10-06T11:22:01.617577+0000","last_fullsized":"2023-10-06T11:22:01.617577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641228+0000","last_change":"2023-10-06T11:22:01.637805+0000","last_active":"2023-10-06T11:22:01.641228+0000","last_peered":"2023-10-06T11:22:01.641228+0000","last_clean":"2023-10-06T11:22:01.641228+0000","last_became_active":"2023-10-06T11:22:01.637404+0000","last_became_peered":"2023-10-06T11:22:01.637404+0000","last_unstale":"2023-10-06T11:22:01.641228+0000","last_undegraded":"2023-10-06T11:22:01.641228+0000","last_fullsized":"2023-10-06T11:22:01.641228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621633+0000","last_change":"2023-10-06T11:21:56.578707+0000","last_active":"2023-10-06T11:22:01.621633+0000","last_peered":"2023-10-06T11:22:01.621633+0000","last_clean":"2023-10-06T11:22:01.621633+0000","last_became_active":"2023-10-06T11:21:56.578411+0000","last_became_peered":"2023-10-06T11:21:56.578411+0000","last_unstale":"2023-10-06T11:22:01.621633+0000","last_undegraded":"2023-10-06T11:22:01.621633+0000","last_fullsized":"2023-10-06T11:22:01.621633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610166+0000","last_change":"2023-10-06T11:21:57.570591+0000","last_active":"2023-10-06T11:22:01.610166+0000","last_peered":"2023-10-06T11:22:01.610166+0000","last_clean":"2023-10-06T11:22:01.610166+0000","last_became_active":"2023-10-06T11:21:57.570166+0000","last_became_peered":"2023-10-06T11:21:57.570166+0000","last_unstale":"2023-10-06T11:22:01.610166+0000","last_undegraded":"2023-10-06T11:22:01.610166+0000","last_fullsized":"2023-10-06T11:22:01.610166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644696+0000","last_change":"2023-10-06T11:22:01.638015+0000","last_active":"2023-10-06T11:22:01.644696+0000","last_peered":"2023-10-06T11:22:01.644696+0000","last_clean":"2023-10-06T11:22:01.644696+0000","last_became_active":"2023-10-06T11:22:01.637477+0000","last_became_peered":"2023-10-06T11:22:01.637477+0000","last_unstale":"2023-10-06T11:22:01.644696+0000","last_undegraded":"2023-10-06T11:22:01.644696+0000","last_fullsized":"2023-10-06T11:22:01.644696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:56.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621240+0000","last_change":"2023-10-06T11:21:56.571616+0000","last_active":"2023-10-06T11:22:01.621240+0000","last_peered":"2023-10-06T11:22:01.621240+0000","last_clean":"2023-10-06T11:22:01.621240+0000","last_became_active":"2023-10-06T11:21:56.571289+0000","last_became_peered":"2023-10-06T11:21:56.571289+0000","last_unstale":"2023-10-06T11:22:01.621240+0000","last_undegraded":"2023-10-06T11:22:01.621240+0000","last_fullsized":"2023-10-06T11:22:01.621240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611658+0000","last_change":"2023-10-06T11:21:57.575085+0000","last_active":"2023-10-06T11:22:01.611658+0000","last_peered":"2023-10-06T11:22:01.611658+0000","last_clean":"2023-10-06T11:22:01.611658+0000","last_became_active":"2023-10-06T11:21:57.574716+0000","last_became_peered":"2023-10-06T11:21:57.574716+0000","last_unstale":"2023-10-06T11:22:01.611658+0000","last_undegraded":"2023-10-06T11:22:01.611658+0000","last_fullsized":"2023-10-06T11:22:01.611658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643969+0000","last_change":"2023-10-06T11:22:01.638673+0000","last_active":"2023-10-06T11:22:01.643969+0000","last_peered":"2023-10-06T11:22:01.643969+0000","last_clean":"2023-10-06T11:22:01.643969+0000","last_became_active":"2023-10-06T11:22:01.635630+0000","last_became_peered":"2023-10-06T11:22:01.635630+0000","last_unstale":"2023-10-06T11:22:01.643969+0000","last_undegraded":"2023-10-06T11:22:01.643969+0000","last_fullsized":"2023-10-06T11:22:01.643969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626800+0000","last_change":"2023-10-06T11:21:56.572349+0000","last_active":"2023-10-06T11:22:01.626800+0000","last_peered":"2023-10-06T11:22:01.626800+0000","last_clean":"2023-10-06T11:22:01.626800+0000","last_became_active":"2023-10-06T11:21:56.571848+0000","last_became_peered":"2023-10-06T11:21:56.571848+0000","last_unstale":"2023-10-06T11:22:01.626800+0000","last_undegraded":"2023-10-06T11:22:01.626800+0000","last_fullsized":"2023-10-06T11:22:01.626800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607694+0000","last_change":"2023-10-06T11:21:57.580595+0000","last_active":"2023-10-06T11:22:01.607694+0000","last_peered":"2023-10-06T11:22:01.607694+0000","last_clean":"2023-10-06T11:22:01.607694+0000","last_became_active":"2023-10-06T11:21:57.580286+0000","last_became_peered":"2023-10-06T11:21:57.580286+0000","last_unstale":"2023-10-06T11:22:01.607694+0000","last_undegraded":"2023-10-06T11:22:01.607694+0000","last_fullsized":"2023-10-06T11:22:01.607694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:04.725222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642661+0000","last_change":"2023-10-06T11:22:01.632283+0000","last_active":"2023-10-06T11:22:01.642661+0000","last_peered":"2023-10-06T11:22:01.642661+0000","last_clean":"2023-10-06T11:22:01.642661+0000","last_became_active":"2023-10-06T11:22:01.628222+0000","last_became_peered":"2023-10-06T11:22:01.628222+0000","last_unstale":"2023-10-06T11:22:01.642661+0000","last_undegraded":"2023-10-06T11:22:01.642661+0000","last_fullsized":"2023-10-06T11:22:01.642661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:32.168166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610453+0000","last_change":"2023-10-06T11:21:56.571031+0000","last_active":"2023-10-06T11:22:01.610453+0000","last_peered":"2023-10-06T11:22:01.610453+0000","last_clean":"2023-10-06T11:22:01.610453+0000","last_became_active":"2023-10-06T11:21:56.570724+0000","last_became_peered":"2023-10-06T11:21:56.570724+0000","last_unstale":"2023-10-06T11:22:01.610453+0000","last_undegraded":"2023-10-06T11:22:01.610453+0000","last_fullsized":"2023-10-06T11:22:01.610453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616088+0000","last_change":"2023-10-06T11:21:57.562757+0000","last_active":"2023-10-06T11:22:01.616088+0000","last_peered":"2023-10-06T11:22:01.616088+0000","last_clean":"2023-10-06T11:22:01.616088+0000","last_became_active":"2023-10-06T11:21:57.562240+0000","last_became_peered":"2023-10-06T11:21:57.562240+0000","last_unstale":"2023-10-06T11:22:01.616088+0000","last_undegraded":"2023-10-06T11:22:01.616088+0000","last_fullsized":"2023-10-06T11:22:01.616088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.636503+0000","last_change":"2023-10-06T11:22:01.635655+0000","last_active":"2023-10-06T11:22:01.636503+0000","last_peered":"2023-10-06T11:22:01.636503+0000","last_clean":"2023-10-06T11:22:01.636503+0000","last_became_active":"2023-10-06T11:22:01.635167+0000","last_became_peered":"2023-10-06T11:22:01.635167+0000","last_unstale":"2023-10-06T11:22:01.636503+0000","last_undegraded":"2023-10-06T11:22:01.636503+0000","last_fullsized":"2023-10-06T11:22:01.636503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:16.080135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617232+0000","last_change":"2023-10-06T11:21:56.569415+0000","last_active":"2023-10-06T11:22:01.617232+0000","last_peered":"2023-10-06T11:22:01.617232+0000","last_clean":"2023-10-06T11:22:01.617232+0000","last_became_active":"2023-10-06T11:21:56.569132+0000","last_became_peered":"2023-10-06T11:21:56.569132+0000","last_unstale":"2023-10-06T11:22:01.617232+0000","last_undegraded":"2023-10-06T11:22:01.617232+0000","last_fullsized":"2023-10-06T11:22:01.617232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627493+0000","last_change":"2023-10-06T11:21:57.566844+0000","last_active":"2023-10-06T11:22:01.627493+0000","last_peered":"2023-10-06T11:22:01.627493+0000","last_clean":"2023-10-06T11:22:01.627493+0000","last_became_active":"2023-10-06T11:21:57.566564+0000","last_became_peered":"2023-10-06T11:21:57.566564+0000","last_unstale":"2023-10-06T11:22:01.627493+0000","last_undegraded":"2023-10-06T11:22:01.627493+0000","last_fullsized":"2023-10-06T11:22:01.627493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640584+0000","last_change":"2023-10-06T11:22:01.639744+0000","last_active":"2023-10-06T11:22:01.640584+0000","last_peered":"2023-10-06T11:22:01.640584+0000","last_clean":"2023-10-06T11:22:01.640584+0000","last_became_active":"2023-10-06T11:22:01.639253+0000","last_became_peered":"2023-10-06T11:22:01.639253+0000","last_unstale":"2023-10-06T11:22:01.640584+0000","last_undegraded":"2023-10-06T11:22:01.640584+0000","last_fullsized":"2023-10-06T11:22:01.640584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615560+0000","last_change":"2023-10-06T11:21:56.574525+0000","last_active":"2023-10-06T11:22:01.615560+0000","last_peered":"2023-10-06T11:22:01.615560+0000","last_clean":"2023-10-06T11:22:01.615560+0000","last_became_active":"2023-10-06T11:21:56.574182+0000","last_became_peered":"2023-10-06T11:21:56.574182+0000","last_unstale":"2023-10-06T11:22:01.615560+0000","last_undegraded":"2023-10-06T11:22:01.615560+0000","last_fullsized":"2023-10-06T11:22:01.615560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620100+0000","last_change":"2023-10-06T11:21:57.564759+0000","last_active":"2023-10-06T11:22:01.620100+0000","last_peered":"2023-10-06T11:22:01.620100+0000","last_clean":"2023-10-06T11:22:01.620100+0000","last_became_active":"2023-10-06T11:21:57.564385+0000","last_became_peered":"2023-10-06T11:21:57.564385+0000","last_unstale":"2023-10-06T11:22:01.620100+0000","last_undegraded":"2023-10-06T11:22:01.620100+0000","last_fullsized":"2023-10-06T11:22:01.620100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615364+0000","last_change":"2023-10-06T11:21:57.567825+0000","last_active":"2023-10-06T11:22:01.615364+0000","last_peered":"2023-10-06T11:22:01.615364+0000","last_clean":"2023-10-06T11:22:01.615364+0000","last_became_active":"2023-10-06T11:21:57.562342+0000","last_became_peered":"2023-10-06T11:21:57.562342+0000","last_unstale":"2023-10-06T11:22:01.615364+0000","last_undegraded":"2023-10-06T11:22:01.615364+0000","last_fullsized":"2023-10-06T11:22:01.615364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627753+0000","last_change":"2023-10-06T11:21:56.573919+0000","last_active":"2023-10-06T11:22:01.627753+0000","last_peered":"2023-10-06T11:22:01.627753+0000","last_clean":"2023-10-06T11:22:01.627753+0000","last_became_active":"2023-10-06T11:21:56.573605+0000","last_became_peered":"2023-10-06T11:21:56.573605+0000","last_unstale":"2023-10-06T11:22:01.627753+0000","last_undegraded":"2023-10-06T11:22:01.627753+0000","last_fullsized":"2023-10-06T11:22:01.627753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639931+0000","last_change":"2023-10-06T11:22:01.637440+0000","last_active":"2023-10-06T11:22:01.639931+0000","last_peered":"2023-10-06T11:22:01.639931+0000","last_clean":"2023-10-06T11:22:01.639931+0000","last_became_active":"2023-10-06T11:22:01.637003+0000","last_became_peered":"2023-10-06T11:22:01.637003+0000","last_unstale":"2023-10-06T11:22:01.639931+0000","last_undegraded":"2023-10-06T11:22:01.639931+0000","last_fullsized":"2023-10-06T11:22:01.639931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.578282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617665+0000","last_change":"2023-10-06T11:21:57.572470+0000","last_active":"2023-10-06T11:22:01.617665+0000","last_peered":"2023-10-06T11:22:01.617665+0000","last_clean":"2023-10-06T11:22:01.617665+0000","last_became_active":"2023-10-06T11:21:57.568699+0000","last_became_peered":"2023-10-06T11:21:57.568699+0000","last_unstale":"2023-10-06T11:22:01.617665+0000","last_undegraded":"2023-10-06T11:22:01.617665+0000","last_fullsized":"2023-10-06T11:22:01.617665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621640+0000","last_change":"2023-10-06T11:21:56.576266+0000","last_active":"2023-10-06T11:22:01.621640+0000","last_peered":"2023-10-06T11:22:01.621640+0000","last_clean":"2023-10-06T11:22:01.621640+0000","last_became_active":"2023-10-06T11:21:56.575879+0000","last_became_peered":"2023-10-06T11:21:56.575879+0000","last_unstale":"2023-10-06T11:22:01.621640+0000","last_undegraded":"2023-10-06T11:22:01.621640+0000","last_fullsized":"2023-10-06T11:22:01.621640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645265+0000","last_change":"2023-10-06T11:22:01.642796+0000","last_active":"2023-10-06T11:22:01.645265+0000","last_peered":"2023-10-06T11:22:01.645265+0000","last_clean":"2023-10-06T11:22:01.645265+0000","last_became_active":"2023-10-06T11:22:01.642102+0000","last_became_peered":"2023-10-06T11:22:01.642102+0000","last_unstale":"2023-10-06T11:22:01.645265+0000","last_undegraded":"2023-10-06T11:22:01.645265+0000","last_fullsized":"2023-10-06T11:22:01.645265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620311+0000","last_change":"2023-10-06T11:21:56.577336+0000","last_active":"2023-10-06T11:22:01.620311+0000","last_peered":"2023-10-06T11:22:01.620311+0000","last_clean":"2023-10-06T11:22:01.620311+0000","last_became_active":"2023-10-06T11:21:56.574934+0000","last_became_peered":"2023-10-06T11:21:56.574934+0000","last_unstale":"2023-10-06T11:22:01.620311+0000","last_undegraded":"2023-10-06T11:22:01.620311+0000","last_fullsized":"2023-10-06T11:22:01.620311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:58.978806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615123+0000","last_change":"2023-10-06T11:21:57.565736+0000","last_active":"2023-10-06T11:22:01.615123+0000","last_peered":"2023-10-06T11:22:01.615123+0000","last_clean":"2023-10-06T11:22:01.615123+0000","last_became_active":"2023-10-06T11:21:57.565078+0000","last_became_peered":"2023-10-06T11:21:57.565078+0000","last_unstale":"2023-10-06T11:22:01.615123+0000","last_undegraded":"2023-10-06T11:22:01.615123+0000","last_fullsized":"2023-10-06T11:22:01.615123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637186+0000","last_change":"2023-10-06T11:22:01.635715+0000","last_active":"2023-10-06T11:22:01.637186+0000","last_peered":"2023-10-06T11:22:01.637186+0000","last_clean":"2023-10-06T11:22:01.637186+0000","last_became_active":"2023-10-06T11:22:01.635213+0000","last_became_peered":"2023-10-06T11:22:01.635213+0000","last_unstale":"2023-10-06T11:22:01.637186+0000","last_undegraded":"2023-10-06T11:22:01.637186+0000","last_fullsized":"2023-10-06T11:22:01.637186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:00.176987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-06T11:22:07.460343+0000","last_change":"2023-10-06T11:21:56.573406+0000","last_active":"2023-10-06T11:22:07.460343+0000","last_peered":"2023-10-06T11:22:07.460343+0000","last_clean":"2023-10-06T11:22:07.460343+0000","last_became_active":"2023-10-06T11:21:56.572858+0000","last_became_peered":"2023-10-06T11:21:56.572858+0000","last_unstale":"2023-10-06T11:22:07.460343+0000","last_undegraded":"2023-10-06T11:22:07.460343+0000","last_fullsized":"2023-10-06T11:22:07.460343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611768+0000","last_change":"2023-10-06T11:21:57.566510+0000","last_active":"2023-10-06T11:22:01.611768+0000","last_peered":"2023-10-06T11:22:01.611768+0000","last_clean":"2023-10-06T11:22:01.611768+0000","last_became_active":"2023-10-06T11:21:57.566168+0000","last_became_peered":"2023-10-06T11:21:57.566168+0000","last_unstale":"2023-10-06T11:22:01.611768+0000","last_undegraded":"2023-10-06T11:22:01.611768+0000","last_fullsized":"2023-10-06T11:22:01.611768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:03.939980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644970+0000","last_change":"2023-10-06T11:22:01.640707+0000","last_active":"2023-10-06T11:22:01.644970+0000","last_peered":"2023-10-06T11:22:01.644970+0000","last_clean":"2023-10-06T11:22:01.644970+0000","last_became_active":"2023-10-06T11:22:01.640294+0000","last_became_peered":"2023-10-06T11:22:01.640294+0000","last_unstale":"2023-10-06T11:22:01.644970+0000","last_undegraded":"2023-10-06T11:22:01.644970+0000","last_fullsized":"2023-10-06T11:22:01.644970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:07.995705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615237+0000","last_change":"2023-10-06T11:21:57.565357+0000","last_active":"2023-10-06T11:22:01.615237+0000","last_peered":"2023-10-06T11:22:01.615237+0000","last_clean":"2023-10-06T11:22:01.615237+0000","last_became_active":"2023-10-06T11:21:57.560887+0000","last_became_peered":"2023-10-06T11:21:57.560887+0000","last_unstale":"2023-10-06T11:22:01.615237+0000","last_undegraded":"2023-10-06T11:22:01.615237+0000","last_fullsized":"2023-10-06T11:22:01.615237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620136+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:22:01.620136+0000","last_peered":"2023-10-06T11:22:01.620136+0000","last_clean":"2023-10-06T11:22:01.620136+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:22:01.620136+0000","last_undegraded":"2023-10-06T11:22:01.620136+0000","last_fullsized":"2023-10-06T11:22:01.620136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621725+0000","last_change":"2023-10-06T11:21:56.569726+0000","last_active":"2023-10-06T11:22:01.621725+0000","last_peered":"2023-10-06T11:22:01.621725+0000","last_clean":"2023-10-06T11:22:01.621725+0000","last_became_active":"2023-10-06T11:21:56.569394+0000","last_became_peered":"2023-10-06T11:21:56.569394+0000","last_unstale":"2023-10-06T11:22:01.621725+0000","last_undegraded":"2023-10-06T11:22:01.621725+0000","last_fullsized":"2023-10-06T11:22:01.621725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640548+0000","last_change":"2023-10-06T11:22:01.637613+0000","last_active":"2023-10-06T11:22:01.640548+0000","last_peered":"2023-10-06T11:22:01.640548+0000","last_clean":"2023-10-06T11:22:01.640548+0000","last_became_active":"2023-10-06T11:22:01.636818+0000","last_became_peered":"2023-10-06T11:22:01.636818+0000","last_unstale":"2023-10-06T11:22:01.640548+0000","last_undegraded":"2023-10-06T11:22:01.640548+0000","last_fullsized":"2023-10-06T11:22:01.640548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626583+0000","last_change":"2023-10-06T11:21:57.556132+0000","last_active":"2023-10-06T11:22:01.626583+0000","last_peered":"2023-10-06T11:22:01.626583+0000","last_clean":"2023-10-06T11:22:01.626583+0000","last_became_active":"2023-10-06T11:21:57.555766+0000","last_became_peered":"2023-10-06T11:21:57.555766+0000","last_unstale":"2023-10-06T11:22:01.626583+0000","last_undegraded":"2023-10-06T11:22:01.626583+0000","last_fullsized":"2023-10-06T11:22:01.626583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620468+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:22:01.620468+0000","last_peered":"2023-10-06T11:22:01.620468+0000","last_clean":"2023-10-06T11:22:01.620468+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:22:01.620468+0000","last_undegraded":"2023-10-06T11:22:01.620468+0000","last_fullsized":"2023-10-06T11:22:01.620468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619168+0000","last_change":"2023-10-06T11:21:56.570891+0000","last_active":"2023-10-06T11:22:01.619168+0000","last_peered":"2023-10-06T11:22:01.619168+0000","last_clean":"2023-10-06T11:22:01.619168+0000","last_became_active":"2023-10-06T11:21:56.570569+0000","last_became_peered":"2023-10-06T11:21:56.570569+0000","last_unstale":"2023-10-06T11:22:01.619168+0000","last_undegraded":"2023-10-06T11:22:01.619168+0000","last_fullsized":"2023-10-06T11:22:01.619168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634770+0000","last_change":"2023-10-06T11:22:01.630420+0000","last_active":"2023-10-06T11:22:01.634770+0000","last_peered":"2023-10-06T11:22:01.634770+0000","last_clean":"2023-10-06T11:22:01.634770+0000","last_became_active":"2023-10-06T11:22:01.629842+0000","last_became_peered":"2023-10-06T11:22:01.629842+0000","last_unstale":"2023-10-06T11:22:01.634770+0000","last_undegraded":"2023-10-06T11:22:01.634770+0000","last_fullsized":"2023-10-06T11:22:01.634770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:49.797072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615458+0000","last_change":"2023-10-06T11:21:57.568394+0000","last_active":"2023-10-06T11:22:01.615458+0000","last_peered":"2023-10-06T11:22:01.615458+0000","last_clean":"2023-10-06T11:22:01.615458+0000","last_became_active":"2023-10-06T11:21:57.568120+0000","last_became_peered":"2023-10-06T11:21:57.568120+0000","last_unstale":"2023-10-06T11:22:01.615458+0000","last_undegraded":"2023-10-06T11:22:01.615458+0000","last_fullsized":"2023-10-06T11:22:01.615458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610919+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:22:01.610919+0000","last_peered":"2023-10-06T11:22:01.610919+0000","last_clean":"2023-10-06T11:22:01.610919+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:22:01.610919+0000","last_undegraded":"2023-10-06T11:22:01.610919+0000","last_fullsized":"2023-10-06T11:22:01.610919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616279+0000","last_change":"2023-10-06T11:21:56.574787+0000","last_active":"2023-10-06T11:22:01.616279+0000","last_peered":"2023-10-06T11:22:01.616279+0000","last_clean":"2023-10-06T11:22:01.616279+0000","last_became_active":"2023-10-06T11:21:56.574471+0000","last_became_peered":"2023-10-06T11:21:56.574471+0000","last_unstale":"2023-10-06T11:22:01.616279+0000","last_undegraded":"2023-10-06T11:22:01.616279+0000","last_fullsized":"2023-10-06T11:22:01.616279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642714+0000","last_change":"2023-10-06T11:22:01.632835+0000","last_active":"2023-10-06T11:22:01.642714+0000","last_peered":"2023-10-06T11:22:01.642714+0000","last_clean":"2023-10-06T11:22:01.642714+0000","last_became_active":"2023-10-06T11:22:01.632391+0000","last_became_peered":"2023-10-06T11:22:01.632391+0000","last_unstale":"2023-10-06T11:22:01.642714+0000","last_undegraded":"2023-10-06T11:22:01.642714+0000","last_fullsized":"2023-10-06T11:22:01.642714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:08.263854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611307+0000","last_change":"2023-10-06T11:21:57.561918+0000","last_active":"2023-10-06T11:22:01.611307+0000","last_peered":"2023-10-06T11:22:01.611307+0000","last_clean":"2023-10-06T11:22:01.611307+0000","last_became_active":"2023-10-06T11:21:57.560917+0000","last_became_peered":"2023-10-06T11:21:57.560917+0000","last_unstale":"2023-10-06T11:22:01.611307+0000","last_undegraded":"2023-10-06T11:22:01.611307+0000","last_fullsized":"2023-10-06T11:22:01.611307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616596+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:22:01.616596+0000","last_peered":"2023-10-06T11:22:01.616596+0000","last_clean":"2023-10-06T11:22:01.616596+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:22:01.616596+0000","last_undegraded":"2023-10-06T11:22:01.616596+0000","last_fullsized":"2023-10-06T11:22:01.616596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:02.262049+0000","last_change":"2023-10-06T11:21:56.573826+0000","last_active":"2023-10-06T11:22:02.262049+0000","last_peered":"2023-10-06T11:22:02.262049+0000","last_clean":"2023-10-06T11:22:02.262049+0000","last_became_active":"2023-10-06T11:21:56.573509+0000","last_became_peered":"2023-10-06T11:21:56.573509+0000","last_unstale":"2023-10-06T11:22:02.262049+0000","last_undegraded":"2023-10-06T11:22:02.262049+0000","last_fullsized":"2023-10-06T11:22:02.262049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634403+0000","last_change":"2023-10-06T11:22:01.631312+0000","last_active":"2023-10-06T11:22:01.634403+0000","last_peered":"2023-10-06T11:22:01.634403+0000","last_clean":"2023-10-06T11:22:01.634403+0000","last_became_active":"2023-10-06T11:22:01.630877+0000","last_became_peered":"2023-10-06T11:22:01.630877+0000","last_unstale":"2023-10-06T11:22:01.634403+0000","last_undegraded":"2023-10-06T11:22:01.634403+0000","last_fullsized":"2023-10-06T11:22:01.634403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:03:28.590751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627610+0000","last_change":"2023-10-06T11:21:57.567907+0000","last_active":"2023-10-06T11:22:01.627610+0000","last_peered":"2023-10-06T11:22:01.627610+0000","last_clean":"2023-10-06T11:22:01.627610+0000","last_became_active":"2023-10-06T11:21:57.567544+0000","last_became_peered":"2023-10-06T11:21:57.567544+0000","last_unstale":"2023-10-06T11:22:01.627610+0000","last_undegraded":"2023-10-06T11:22:01.627610+0000","last_fullsized":"2023-10-06T11:22:01.627610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.625024+0000","last_change":"2023-10-06T11:21:56.571499+0000","last_active":"2023-10-06T11:22:01.625024+0000","last_peered":"2023-10-06T11:22:01.625024+0000","last_clean":"2023-10-06T11:22:01.625024+0000","last_became_active":"2023-10-06T11:21:56.571176+0000","last_became_peered":"2023-10-06T11:21:56.571176+0000","last_unstale":"2023-10-06T11:22:01.625024+0000","last_undegraded":"2023-10-06T11:22:01.625024+0000","last_fullsized":"2023-10-06T11:22:01.625024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616847+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:22:01.616847+0000","last_peered":"2023-10-06T11:22:01.616847+0000","last_clean":"2023-10-06T11:22:01.616847+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:22:01.616847+0000","last_undegraded":"2023-10-06T11:22:01.616847+0000","last_fullsized":"2023-10-06T11:22:01.616847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614070+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:22:01.614070+0000","last_peered":"2023-10-06T11:22:01.614070+0000","last_clean":"2023-10-06T11:22:01.614070+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:22:01.614070+0000","last_undegraded":"2023-10-06T11:22:01.614070+0000","last_fullsized":"2023-10-06T11:22:01.614070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635892+0000","last_change":"2023-10-06T11:22:01.632615+0000","last_active":"2023-10-06T11:22:01.635892+0000","last_peered":"2023-10-06T11:22:01.635892+0000","last_clean":"2023-10-06T11:22:01.635892+0000","last_became_active":"2023-10-06T11:22:01.632167+0000","last_became_peered":"2023-10-06T11:22:01.632167+0000","last_unstale":"2023-10-06T11:22:01.635892+0000","last_undegraded":"2023-10-06T11:22:01.635892+0000","last_fullsized":"2023-10-06T11:22:01.635892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:35.069167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614553+0000","last_change":"2023-10-06T11:21:57.558911+0000","last_active":"2023-10-06T11:22:01.614553+0000","last_peered":"2023-10-06T11:22:01.614553+0000","last_clean":"2023-10-06T11:22:01.614553+0000","last_became_active":"2023-10-06T11:21:57.555525+0000","last_became_peered":"2023-10-06T11:21:57.555525+0000","last_unstale":"2023-10-06T11:22:01.614553+0000","last_undegraded":"2023-10-06T11:22:01.614553+0000","last_fullsized":"2023-10-06T11:22:01.614553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620129+0000","last_change":"2023-10-06T11:21:56.566441+0000","last_active":"2023-10-06T11:22:01.620129+0000","last_peered":"2023-10-06T11:22:01.620129+0000","last_clean":"2023-10-06T11:22:01.620129+0000","last_became_active":"2023-10-06T11:21:56.566070+0000","last_became_peered":"2023-10-06T11:21:56.566070+0000","last_unstale":"2023-10-06T11:22:01.620129+0000","last_undegraded":"2023-10-06T11:22:01.620129+0000","last_fullsized":"2023-10-06T11:22:01.620129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616748+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:22:01.616748+0000","last_peered":"2023-10-06T11:22:01.616748+0000","last_clean":"2023-10-06T11:22:01.616748+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:22:01.616748+0000","last_undegraded":"2023-10-06T11:22:01.616748+0000","last_fullsized":"2023-10-06T11:22:01.616748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637816+0000","last_change":"2023-10-06T11:22:01.632881+0000","last_active":"2023-10-06T11:22:01.637816+0000","last_peered":"2023-10-06T11:22:01.637816+0000","last_clean":"2023-10-06T11:22:01.637816+0000","last_became_active":"2023-10-06T11:22:01.632330+0000","last_became_peered":"2023-10-06T11:22:01.632330+0000","last_unstale":"2023-10-06T11:22:01.637816+0000","last_undegraded":"2023-10-06T11:22:01.637816+0000","last_fullsized":"2023-10-06T11:22:01.637816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:38.278034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619088+0000","last_change":"2023-10-06T11:21:57.565393+0000","last_active":"2023-10-06T11:22:01.619088+0000","last_peered":"2023-10-06T11:22:01.619088+0000","last_clean":"2023-10-06T11:22:01.619088+0000","last_became_active":"2023-10-06T11:21:57.565128+0000","last_became_peered":"2023-10-06T11:21:57.565128+0000","last_unstale":"2023-10-06T11:22:01.619088+0000","last_undegraded":"2023-10-06T11:22:01.619088+0000","last_fullsized":"2023-10-06T11:22:01.619088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611906+0000","last_change":"2023-10-06T11:21:56.573109+0000","last_active":"2023-10-06T11:22:01.611906+0000","last_peered":"2023-10-06T11:22:01.611906+0000","last_clean":"2023-10-06T11:22:01.611906+0000","last_became_active":"2023-10-06T11:21:56.572783+0000","last_became_peered":"2023-10-06T11:21:56.572783+0000","last_unstale":"2023-10-06T11:22:01.611906+0000","last_undegraded":"2023-10-06T11:22:01.611906+0000","last_fullsized":"2023-10-06T11:22:01.611906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620654+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:22:01.620654+0000","last_peered":"2023-10-06T11:22:01.620654+0000","last_clean":"2023-10-06T11:22:01.620654+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:22:01.620654+0000","last_undegraded":"2023-10-06T11:22:01.620654+0000","last_fullsized":"2023-10-06T11:22:01.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643525+0000","last_change":"2023-10-06T11:22:01.641973+0000","last_active":"2023-10-06T11:22:01.643525+0000","last_peered":"2023-10-06T11:22:01.643525+0000","last_clean":"2023-10-06T11:22:01.643525+0000","last_became_active":"2023-10-06T11:22:01.641394+0000","last_became_peered":"2023-10-06T11:22:01.641394+0000","last_unstale":"2023-10-06T11:22:01.643525+0000","last_undegraded":"2023-10-06T11:22:01.643525+0000","last_fullsized":"2023-10-06T11:22:01.643525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620183+0000","last_change":"2023-10-06T11:21:57.564937+0000","last_active":"2023-10-06T11:22:01.620183+0000","last_peered":"2023-10-06T11:22:01.620183+0000","last_clean":"2023-10-06T11:22:01.620183+0000","last_became_active":"2023-10-06T11:21:57.564603+0000","last_became_peered":"2023-10-06T11:21:57.564603+0000","last_unstale":"2023-10-06T11:22:01.620183+0000","last_undegraded":"2023-10-06T11:22:01.620183+0000","last_fullsized":"2023-10-06T11:22:01.620183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615864+0000","last_change":"2023-10-06T11:21:56.567945+0000","last_active":"2023-10-06T11:22:01.615864+0000","last_peered":"2023-10-06T11:22:01.615864+0000","last_clean":"2023-10-06T11:22:01.615864+0000","last_became_active":"2023-10-06T11:21:56.567592+0000","last_became_peered":"2023-10-06T11:21:56.567592+0000","last_unstale":"2023-10-06T11:22:01.615864+0000","last_undegraded":"2023-10-06T11:22:01.615864+0000","last_fullsized":"2023-10-06T11:22:01.615864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616944+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:22:01.616944+0000","last_peered":"2023-10-06T11:22:01.616944+0000","last_clean":"2023-10-06T11:22:01.616944+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:22:01.616944+0000","last_undegraded":"2023-10-06T11:22:01.616944+0000","last_fullsized":"2023-10-06T11:22:01.616944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644814+0000","last_change":"2023-10-06T11:22:01.639193+0000","last_active":"2023-10-06T11:22:01.644814+0000","last_peered":"2023-10-06T11:22:01.644814+0000","last_clean":"2023-10-06T11:22:01.644814+0000","last_became_active":"2023-10-06T11:22:01.638730+0000","last_became_peered":"2023-10-06T11:22:01.638730+0000","last_unstale":"2023-10-06T11:22:01.644814+0000","last_undegraded":"2023-10-06T11:22:01.644814+0000","last_fullsized":"2023-10-06T11:22:01.644814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614448+0000","last_change":"2023-10-06T11:21:57.565173+0000","last_active":"2023-10-06T11:22:01.614448+0000","last_peered":"2023-10-06T11:22:01.614448+0000","last_clean":"2023-10-06T11:22:01.614448+0000","last_became_active":"2023-10-06T11:21:57.564865+0000","last_became_peered":"2023-10-06T11:21:57.564865+0000","last_unstale":"2023-10-06T11:22:01.614448+0000","last_undegraded":"2023-10-06T11:22:01.614448+0000","last_fullsized":"2023-10-06T11:22:01.614448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616901+0000","last_change":"2023-10-06T11:21:56.570015+0000","last_active":"2023-10-06T11:22:01.616901+0000","last_peered":"2023-10-06T11:22:01.616901+0000","last_clean":"2023-10-06T11:22:01.616901+0000","last_became_active":"2023-10-06T11:21:56.569530+0000","last_became_peered":"2023-10-06T11:21:56.569530+0000","last_unstale":"2023-10-06T11:22:01.616901+0000","last_undegraded":"2023-10-06T11:22:01.616901+0000","last_fullsized":"2023-10-06T11:22:01.616901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642672+0000","last_change":"2023-10-06T11:22:01.632016+0000","last_active":"2023-10-06T11:22:01.642672+0000","last_peered":"2023-10-06T11:22:01.642672+0000","last_clean":"2023-10-06T11:22:01.642672+0000","last_became_active":"2023-10-06T11:22:01.629544+0000","last_became_peered":"2023-10-06T11:22:01.629544+0000","last_unstale":"2023-10-06T11:22:01.642672+0000","last_undegraded":"2023-10-06T11:22:01.642672+0000","last_fullsized":"2023-10-06T11:22:01.642672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:20.535717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611336+0000","last_change":"2023-10-06T11:21:57.566827+0000","last_active":"2023-10-06T11:22:01.611336+0000","last_peered":"2023-10-06T11:22:01.611336+0000","last_clean":"2023-10-06T11:22:01.611336+0000","last_became_active":"2023-10-06T11:21:57.566463+0000","last_became_peered":"2023-10-06T11:21:57.566463+0000","last_unstale":"2023-10-06T11:22:01.611336+0000","last_undegraded":"2023-10-06T11:22:01.611336+0000","last_fullsized":"2023-10-06T11:22:01.611336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611609+0000","last_change":"2023-10-06T11:21:56.573082+0000","last_active":"2023-10-06T11:22:01.611609+0000","last_peered":"2023-10-06T11:22:01.611609+0000","last_clean":"2023-10-06T11:22:01.611609+0000","last_became_active":"2023-10-06T11:21:56.572363+0000","last_became_peered":"2023-10-06T11:21:56.572363+0000","last_unstale":"2023-10-06T11:22:01.611609+0000","last_undegraded":"2023-10-06T11:22:01.611609+0000","last_fullsized":"2023-10-06T11:22:01.611609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.628822+0000","last_change":"2023-10-06T11:22:01.626277+0000","last_active":"2023-10-06T11:22:01.628822+0000","last_peered":"2023-10-06T11:22:01.628822+0000","last_clean":"2023-10-06T11:22:01.628822+0000","last_became_active":"2023-10-06T11:22:01.625806+0000","last_became_peered":"2023-10-06T11:22:01.625806+0000","last_unstale":"2023-10-06T11:22:01.628822+0000","last_undegraded":"2023-10-06T11:22:01.628822+0000","last_fullsized":"2023-10-06T11:22:01.628822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611575+0000","last_change":"2023-10-06T11:21:57.564883+0000","last_active":"2023-10-06T11:22:01.611575+0000","last_peered":"2023-10-06T11:22:01.611575+0000","last_clean":"2023-10-06T11:22:01.611575+0000","last_became_active":"2023-10-06T11:21:57.564586+0000","last_became_peered":"2023-10-06T11:21:57.564586+0000","last_unstale":"2023-10-06T11:22:01.611575+0000","last_undegraded":"2023-10-06T11:22:01.611575+0000","last_fullsized":"2023-10-06T11:22:01.611575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:12.024410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621041+0000","last_change":"2023-10-06T11:21:56.576790+0000","last_active":"2023-10-06T11:22:01.621041+0000","last_peered":"2023-10-06T11:22:01.621041+0000","last_clean":"2023-10-06T11:22:01.621041+0000","last_became_active":"2023-10-06T11:21:56.576463+0000","last_became_peered":"2023-10-06T11:21:56.576463+0000","last_unstale":"2023-10-06T11:22:01.621041+0000","last_undegraded":"2023-10-06T11:22:01.621041+0000","last_fullsized":"2023-10-06T11:22:01.621041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643771+0000","last_change":"2023-10-06T11:22:01.639619+0000","last_active":"2023-10-06T11:22:01.643771+0000","last_peered":"2023-10-06T11:22:01.643771+0000","last_clean":"2023-10-06T11:22:01.643771+0000","last_became_active":"2023-10-06T11:22:01.638615+0000","last_became_peered":"2023-10-06T11:22:01.638615+0000","last_unstale":"2023-10-06T11:22:01.643771+0000","last_undegraded":"2023-10-06T11:22:01.643771+0000","last_fullsized":"2023-10-06T11:22:01.643771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612709+0000","last_change":"2023-10-06T11:21:57.577150+0000","last_active":"2023-10-06T11:22:01.612709+0000","last_peered":"2023-10-06T11:22:01.612709+0000","last_clean":"2023-10-06T11:22:01.612709+0000","last_became_active":"2023-10-06T11:21:57.576632+0000","last_became_peered":"2023-10-06T11:21:57.576632+0000","last_unstale":"2023-10-06T11:22:01.612709+0000","last_undegraded":"2023-10-06T11:22:01.612709+0000","last_fullsized":"2023-10-06T11:22:01.612709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:32.416966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617372+0000","last_change":"2023-10-06T11:21:56.575818+0000","last_active":"2023-10-06T11:22:01.617372+0000","last_peered":"2023-10-06T11:22:01.617372+0000","last_clean":"2023-10-06T11:22:01.617372+0000","last_became_active":"2023-10-06T11:21:56.575460+0000","last_became_peered":"2023-10-06T11:21:56.575460+0000","last_unstale":"2023-10-06T11:22:01.617372+0000","last_undegraded":"2023-10-06T11:22:01.617372+0000","last_fullsized":"2023-10-06T11:22:01.617372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634446+0000","last_change":"2023-10-06T11:22:01.632266+0000","last_active":"2023-10-06T11:22:01.634446+0000","last_peered":"2023-10-06T11:22:01.634446+0000","last_clean":"2023-10-06T11:22:01.634446+0000","last_became_active":"2023-10-06T11:22:01.631851+0000","last_became_peered":"2023-10-06T11:22:01.631851+0000","last_unstale":"2023-10-06T11:22:01.634446+0000","last_undegraded":"2023-10-06T11:22:01.634446+0000","last_fullsized":"2023-10-06T11:22:01.634446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:11.811755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626480+0000","last_change":"2023-10-06T11:21:57.561599+0000","last_active":"2023-10-06T11:22:01.626480+0000","last_peered":"2023-10-06T11:22:01.626480+0000","last_clean":"2023-10-06T11:22:01.626480+0000","last_became_active":"2023-10-06T11:21:57.561280+0000","last_became_peered":"2023-10-06T11:21:57.561280+0000","last_unstale":"2023-10-06T11:22:01.626480+0000","last_undegraded":"2023-10-06T11:22:01.626480+0000","last_fullsized":"2023-10-06T11:22:01.626480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621119+0000","last_change":"2023-10-06T11:21:56.566817+0000","last_active":"2023-10-06T11:22:01.621119+0000","last_peered":"2023-10-06T11:22:01.621119+0000","last_clean":"2023-10-06T11:22:01.621119+0000","last_became_active":"2023-10-06T11:21:56.566466+0000","last_became_peered":"2023-10-06T11:21:56.566466+0000","last_unstale":"2023-10-06T11:22:01.621119+0000","last_undegraded":"2023-10-06T11:22:01.621119+0000","last_fullsized":"2023-10-06T11:22:01.621119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.632874+0000","last_change":"2023-10-06T11:22:01.628287+0000","last_active":"2023-10-06T11:22:01.632874+0000","last_peered":"2023-10-06T11:22:01.632874+0000","last_clean":"2023-10-06T11:22:01.632874+0000","last_became_active":"2023-10-06T11:22:01.627842+0000","last_became_peered":"2023-10-06T11:22:01.627842+0000","last_unstale":"2023-10-06T11:22:01.632874+0000","last_undegraded":"2023-10-06T11:22:01.632874+0000","last_fullsized":"2023-10-06T11:22:01.632874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:42.027969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620876+0000","last_change":"2023-10-06T11:21:57.572846+0000","last_active":"2023-10-06T11:22:01.620876+0000","last_peered":"2023-10-06T11:22:01.620876+0000","last_clean":"2023-10-06T11:22:01.620876+0000","last_became_active":"2023-10-06T11:21:57.572567+0000","last_became_peered":"2023-10-06T11:21:57.572567+0000","last_unstale":"2023-10-06T11:22:01.620876+0000","last_undegraded":"2023-10-06T11:22:01.620876+0000","last_fullsized":"2023-10-06T11:22:01.620876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620369+0000","last_change":"2023-10-06T11:21:56.575220+0000","last_active":"2023-10-06T11:22:01.620369+0000","last_peered":"2023-10-06T11:22:01.620369+0000","last_clean":"2023-10-06T11:22:01.620369+0000","last_became_active":"2023-10-06T11:21:56.574557+0000","last_became_peered":"2023-10-06T11:21:56.574557+0000","last_unstale":"2023-10-06T11:22:01.620369+0000","last_undegraded":"2023-10-06T11:22:01.620369+0000","last_fullsized":"2023-10-06T11:22:01.620369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639894+0000","last_change":"2023-10-06T11:22:01.637484+0000","last_active":"2023-10-06T11:22:01.639894+0000","last_peered":"2023-10-06T11:22:01.639894+0000","last_clean":"2023-10-06T11:22:01.639894+0000","last_became_active":"2023-10-06T11:22:01.637045+0000","last_became_peered":"2023-10-06T11:22:01.637045+0000","last_unstale":"2023-10-06T11:22:01.639894+0000","last_undegraded":"2023-10-06T11:22:01.639894+0000","last_fullsized":"2023-10-06T11:22:01.639894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:36.472691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610908+0000","last_change":"2023-10-06T11:21:57.562666+0000","last_active":"2023-10-06T11:22:01.610908+0000","last_peered":"2023-10-06T11:22:01.610908+0000","last_clean":"2023-10-06T11:22:01.610908+0000","last_became_active":"2023-10-06T11:21:57.562384+0000","last_became_peered":"2023-10-06T11:21:57.562384+0000","last_unstale":"2023-10-06T11:22:01.610908+0000","last_undegraded":"2023-10-06T11:22:01.610908+0000","last_fullsized":"2023-10-06T11:22:01.610908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:03.832047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619654+0000","last_change":"2023-10-06T11:21:56.578885+0000","last_active":"2023-10-06T11:22:01.619654+0000","last_peered":"2023-10-06T11:22:01.619654+0000","last_clean":"2023-10-06T11:22:01.619654+0000","last_became_active":"2023-10-06T11:21:56.578589+0000","last_became_peered":"2023-10-06T11:21:56.578589+0000","last_unstale":"2023-10-06T11:22:01.619654+0000","last_undegraded":"2023-10-06T11:22:01.619654+0000","last_fullsized":"2023-10-06T11:22:01.619654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:31.263377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.629203+0000","last_change":"2023-10-06T11:22:01.626674+0000","last_active":"2023-10-06T11:22:01.629203+0000","last_peered":"2023-10-06T11:22:01.629203+0000","last_clean":"2023-10-06T11:22:01.629203+0000","last_became_active":"2023-10-06T11:22:01.626199+0000","last_became_peered":"2023-10-06T11:22:01.626199+0000","last_unstale":"2023-10-06T11:22:01.629203+0000","last_undegraded":"2023-10-06T11:22:01.629203+0000","last_fullsized":"2023-10-06T11:22:01.629203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:07.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619746+0000","last_change":"2023-10-06T11:21:57.568271+0000","last_active":"2023-10-06T11:22:01.619746+0000","last_peered":"2023-10-06T11:22:01.619746+0000","last_clean":"2023-10-06T11:22:01.619746+0000","last_became_active":"2023-10-06T11:21:57.567943+0000","last_became_peered":"2023-10-06T11:21:57.567943+0000","last_unstale":"2023-10-06T11:22:01.619746+0000","last_undegraded":"2023-10-06T11:22:01.619746+0000","last_fullsized":"2023-10-06T11:22:01.619746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619520+0000","last_change":"2023-10-06T11:21:56.579019+0000","last_active":"2023-10-06T11:22:01.619520+0000","last_peered":"2023-10-06T11:22:01.619520+0000","last_clean":"2023-10-06T11:22:01.619520+0000","last_became_active":"2023-10-06T11:21:56.578659+0000","last_became_peered":"2023-10-06T11:21:56.578659+0000","last_unstale":"2023-10-06T11:22:01.619520+0000","last_undegraded":"2023-10-06T11:22:01.619520+0000","last_fullsized":"2023-10-06T11:22:01.619520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643049+0000","last_change":"2023-10-06T11:22:01.637818+0000","last_active":"2023-10-06T11:22:01.643049+0000","last_peered":"2023-10-06T11:22:01.643049+0000","last_clean":"2023-10-06T11:22:01.643049+0000","last_became_active":"2023-10-06T11:22:01.637325+0000","last_became_peered":"2023-10-06T11:22:01.637325+0000","last_unstale":"2023-10-06T11:22:01.643049+0000","last_undegraded":"2023-10-06T11:22:01.643049+0000","last_fullsized":"2023-10-06T11:22:01.643049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641080+0000","last_change":"2023-10-06T11:22:01.636779+0000","last_active":"2023-10-06T11:22:01.641080+0000","last_peered":"2023-10-06T11:22:01.641080+0000","last_clean":"2023-10-06T11:22:01.641080+0000","last_became_active":"2023-10-06T11:22:01.636344+0000","last_became_peered":"2023-10-06T11:22:01.636344+0000","last_unstale":"2023-10-06T11:22:01.641080+0000","last_undegraded":"2023-10-06T11:22:01.641080+0000","last_fullsized":"2023-10-06T11:22:01.641080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620730+0000","last_change":"2023-10-06T11:21:56.575584+0000","last_active":"2023-10-06T11:22:01.620730+0000","last_peered":"2023-10-06T11:22:01.620730+0000","last_clean":"2023-10-06T11:22:01.620730+0000","last_became_active":"2023-10-06T11:21:56.575201+0000","last_became_peered":"2023-10-06T11:21:56.575201+0000","last_unstale":"2023-10-06T11:22:01.620730+0000","last_undegraded":"2023-10-06T11:22:01.620730+0000","last_fullsized":"2023-10-06T11:22:01.620730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616376+0000","last_change":"2023-10-06T11:21:57.568259+0000","last_active":"2023-10-06T11:22:01.616376+0000","last_peered":"2023-10-06T11:22:01.616376+0000","last_clean":"2023-10-06T11:22:01.616376+0000","last_became_active":"2023-10-06T11:21:57.567958+0000","last_became_peered":"2023-10-06T11:21:57.567958+0000","last_unstale":"2023-10-06T11:22:01.616376+0000","last_undegraded":"2023-10-06T11:22:01.616376+0000","last_fullsized":"2023-10-06T11:22:01.616376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637658+0000","last_change":"2023-10-06T11:22:01.635766+0000","last_active":"2023-10-06T11:22:01.637658+0000","last_peered":"2023-10-06T11:22:01.637658+0000","last_clean":"2023-10-06T11:22:01.637658+0000","last_became_active":"2023-10-06T11:22:01.635314+0000","last_became_peered":"2023-10-06T11:22:01.635314+0000","last_unstale":"2023-10-06T11:22:01.637658+0000","last_undegraded":"2023-10-06T11:22:01.637658+0000","last_fullsized":"2023-10-06T11:22:01.637658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:33:19.459200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619865+0000","last_change":"2023-10-06T11:21:56.579444+0000","last_active":"2023-10-06T11:22:01.619865+0000","last_peered":"2023-10-06T11:22:01.619865+0000","last_clean":"2023-10-06T11:22:01.619865+0000","last_became_active":"2023-10-06T11:21:56.578427+0000","last_became_peered":"2023-10-06T11:21:56.578427+0000","last_unstale":"2023-10-06T11:22:01.619865+0000","last_undegraded":"2023-10-06T11:22:01.619865+0000","last_fullsized":"2023-10-06T11:22:01.619865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:21.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627696+0000","last_change":"2023-10-06T11:21:57.569150+0000","last_active":"2023-10-06T11:22:01.627696+0000","last_peered":"2023-10-06T11:22:01.627696+0000","last_clean":"2023-10-06T11:22:01.627696+0000","last_became_active":"2023-10-06T11:21:57.568883+0000","last_became_peered":"2023-10-06T11:21:57.568883+0000","last_unstale":"2023-10-06T11:22:01.627696+0000","last_undegraded":"2023-10-06T11:22:01.627696+0000","last_fullsized":"2023-10-06T11:22:01.627696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626255+0000","last_change":"2023-10-06T11:21:56.572563+0000","last_active":"2023-10-06T11:22:01.626255+0000","last_peered":"2023-10-06T11:22:01.626255+0000","last_clean":"2023-10-06T11:22:01.626255+0000","last_became_active":"2023-10-06T11:21:56.571525+0000","last_became_peered":"2023-10-06T11:21:56.571525+0000","last_unstale":"2023-10-06T11:22:01.626255+0000","last_undegraded":"2023-10-06T11:22:01.626255+0000","last_fullsized":"2023-10-06T11:22:01.626255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621535+0000","last_change":"2023-10-06T11:21:57.569979+0000","last_active":"2023-10-06T11:22:01.621535+0000","last_peered":"2023-10-06T11:22:01.621535+0000","last_clean":"2023-10-06T11:22:01.621535+0000","last_became_active":"2023-10-06T11:21:57.569669+0000","last_became_peered":"2023-10-06T11:21:57.569669+0000","last_unstale":"2023-10-06T11:22:01.621535+0000","last_undegraded":"2023-10-06T11:22:01.621535+0000","last_fullsized":"2023-10-06T11:22:01.621535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.638165+0000","last_change":"2023-10-06T11:22:01.636951+0000","last_active":"2023-10-06T11:22:01.638165+0000","last_peered":"2023-10-06T11:22:01.638165+0000","last_clean":"2023-10-06T11:22:01.638165+0000","last_became_active":"2023-10-06T11:22:01.636534+0000","last_became_peered":"2023-10-06T11:22:01.636534+0000","last_unstale":"2023-10-06T11:22:01.638165+0000","last_undegraded":"2023-10-06T11:22:01.638165+0000","last_fullsized":"2023-10-06T11:22:01.638165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621855+0000","last_change":"2023-10-06T11:21:56.579219+0000","last_active":"2023-10-06T11:22:01.621855+0000","last_peered":"2023-10-06T11:22:01.621855+0000","last_clean":"2023-10-06T11:22:01.621855+0000","last_became_active":"2023-10-06T11:21:56.578100+0000","last_became_peered":"2023-10-06T11:21:56.578100+0000","last_unstale":"2023-10-06T11:22:01.621855+0000","last_undegraded":"2023-10-06T11:22:01.621855+0000","last_fullsized":"2023-10-06T11:22:01.621855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616499+0000","last_change":"2023-10-06T11:21:57.580696+0000","last_active":"2023-10-06T11:22:01.616499+0000","last_peered":"2023-10-06T11:22:01.616499+0000","last_clean":"2023-10-06T11:22:01.616499+0000","last_became_active":"2023-10-06T11:21:57.580262+0000","last_became_peered":"2023-10-06T11:21:57.580262+0000","last_unstale":"2023-10-06T11:22:01.616499+0000","last_undegraded":"2023-10-06T11:22:01.616499+0000","last_fullsized":"2023-10-06T11:22:01.616499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635928+0000","last_change":"2023-10-06T11:22:01.632977+0000","last_active":"2023-10-06T11:22:01.635928+0000","last_peered":"2023-10-06T11:22:01.635928+0000","last_clean":"2023-10-06T11:22:01.635928+0000","last_became_active":"2023-10-06T11:22:01.632522+0000","last_became_peered":"2023-10-06T11:22:01.632522+0000","last_unstale":"2023-10-06T11:22:01.635928+0000","last_undegraded":"2023-10-06T11:22:01.635928+0000","last_fullsized":"2023-10-06T11:22:01.635928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620591+0000","last_change":"2023-10-06T11:21:57.570931+0000","last_active":"2023-10-06T11:22:01.620591+0000","last_peered":"2023-10-06T11:22:01.620591+0000","last_clean":"2023-10-06T11:22:01.620591+0000","last_became_active":"2023-10-06T11:21:57.570619+0000","last_became_peered":"2023-10-06T11:21:57.570619+0000","last_unstale":"2023-10-06T11:22:01.620591+0000","last_undegraded":"2023-10-06T11:22:01.620591+0000","last_fullsized":"2023-10-06T11:22:01.620591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615615+0000","last_change":"2023-10-06T11:21:57.572767+0000","last_active":"2023-10-06T11:22:01.615615+0000","last_peered":"2023-10-06T11:22:01.615615+0000","last_clean":"2023-10-06T11:22:01.615615+0000","last_became_active":"2023-10-06T11:21:57.572371+0000","last_became_peered":"2023-10-06T11:21:57.572371+0000","last_unstale":"2023-10-06T11:22:01.615615+0000","last_undegraded":"2023-10-06T11:22:01.615615+0000","last_fullsized":"2023-10-06T11:22:01.615615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610290+0000","last_change":"2023-10-06T11:21:57.571303+0000","last_active":"2023-10-06T11:22:01.610290+0000","last_peered":"2023-10-06T11:22:01.610290+0000","last_clean":"2023-10-06T11:22:01.610290+0000","last_became_active":"2023-10-06T11:21:57.570978+0000","last_became_peered":"2023-10-06T11:21:57.570978+0000","last_unstale":"2023-10-06T11:22:01.610290+0000","last_undegraded":"2023-10-06T11:22:01.610290+0000","last_fullsized":"2023-10-06T11:22:01.610290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617222+0000","last_change":"2023-10-06T11:21:57.569031+0000","last_active":"2023-10-06T11:22:01.617222+0000","last_peered":"2023-10-06T11:22:01.617222+0000","last_clean":"2023-10-06T11:22:01.617222+0000","last_became_active":"2023-10-06T11:21:57.568734+0000","last_became_peered":"2023-10-06T11:21:57.568734+0000","last_unstale":"2023-10-06T11:22:01.617222+0000","last_undegraded":"2023-10-06T11:22:01.617222+0000","last_fullsized":"2023-10-06T11:22:01.617222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619002+0000","last_change":"2023-10-06T11:21:57.579632+0000","last_active":"2023-10-06T11:22:01.619002+0000","last_peered":"2023-10-06T11:22:01.619002+0000","last_clean":"2023-10-06T11:22:01.619002+0000","last_became_active":"2023-10-06T11:21:57.567143+0000","last_became_peered":"2023-10-06T11:21:57.567143+0000","last_unstale":"2023-10-06T11:22:01.619002+0000","last_undegraded":"2023-10-06T11:22:01.619002+0000","last_fullsized":"2023-10-06T11:22:01.619002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621442+0000","last_change":"2023-10-06T11:21:57.569847+0000","last_active":"2023-10-06T11:22:01.621442+0000","last_peered":"2023-10-06T11:22:01.621442+0000","last_clean":"2023-10-06T11:22:01.621442+0000","last_became_active":"2023-10-06T11:21:57.569571+0000","last_became_peered":"2023-10-06T11:21:57.569571+0000","last_unstale":"2023-10-06T11:22:01.621442+0000","last_undegraded":"2023-10-06T11:22:01.621442+0000","last_fullsized":"2023-10-06T11:22:01.621442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621862+0000","last_change":"2023-10-06T11:21:57.562969+0000","last_active":"2023-10-06T11:22:01.621862+0000","last_peered":"2023-10-06T11:22:01.621862+0000","last_clean":"2023-10-06T11:22:01.621862+0000","last_became_active":"2023-10-06T11:21:57.562623+0000","last_became_peered":"2023-10-06T11:21:57.562623+0000","last_unstale":"2023-10-06T11:22:01.621862+0000","last_undegraded":"2023-10-06T11:22:01.621862+0000","last_fullsized":"2023-10-06T11:22:01.621862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614335+0000","last_change":"2023-10-06T11:21:57.571639+0000","last_active":"2023-10-06T11:22:01.614335+0000","last_peered":"2023-10-06T11:22:01.614335+0000","last_clean":"2023-10-06T11:22:01.614335+0000","last_became_active":"2023-10-06T11:21:57.571347+0000","last_became_peered":"2023-10-06T11:21:57.571347+0000","last_unstale":"2023-10-06T11:22:01.614335+0000","last_undegraded":"2023-10-06T11:22:01.614335+0000","last_fullsized":"2023-10-06T11:22:01.614335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616721+0000","last_change":"2023-10-06T11:21:57.576904+0000","last_active":"2023-10-06T11:22:01.616721+0000","last_peered":"2023-10-06T11:22:01.616721+0000","last_clean":"2023-10-06T11:22:01.616721+0000","last_became_active":"2023-10-06T11:21:57.576570+0000","last_became_peered":"2023-10-06T11:21:57.576570+0000","last_unstale":"2023-10-06T11:22:01.616721+0000","last_undegraded":"2023-10-06T11:22:01.616721+0000","last_fullsized":"2023-10-06T11:22:01.616721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615508+0000","last_change":"2023-10-06T11:21:57.572666+0000","last_active":"2023-10-06T11:22:01.615508+0000","last_peered":"2023-10-06T11:22:01.615508+0000","last_clean":"2023-10-06T11:22:01.615508+0000","last_became_active":"2023-10-06T11:21:57.569109+0000","last_became_peered":"2023-10-06T11:21:57.569109+0000","last_unstale":"2023-10-06T11:22:01.615508+0000","last_undegraded":"2023-10-06T11:22:01.615508+0000","last_fullsized":"2023-10-06T11:22:01.615508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626091+0000","last_change":"2023-10-06T11:21:57.566406+0000","last_active":"2023-10-06T11:22:01.626091+0000","last_peered":"2023-10-06T11:22:01.626091+0000","last_clean":"2023-10-06T11:22:01.626091+0000","last_became_active":"2023-10-06T11:21:57.565954+0000","last_became_peered":"2023-10-06T11:21:57.565954+0000","last_unstale":"2023-10-06T11:22:01.626091+0000","last_undegraded":"2023-10-06T11:22:01.626091+0000","last_fullsized":"2023-10-06T11:22:01.626091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621753+0000","last_change":"2023-10-06T11:21:57.576914+0000","last_active":"2023-10-06T11:22:01.621753+0000","last_peered":"2023-10-06T11:22:01.621753+0000","last_clean":"2023-10-06T11:22:01.621753+0000","last_became_active":"2023-10-06T11:21:57.576444+0000","last_became_peered":"2023-10-06T11:21:57.576444+0000","last_unstale":"2023-10-06T11:22:01.621753+0000","last_undegraded":"2023-10-06T11:22:01.621753+0000","last_fullsized":"2023-10-06T11:22:01.621753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612641+0000","last_change":"2023-10-06T11:21:57.565842+0000","last_active":"2023-10-06T11:22:01.612641+0000","last_peered":"2023-10-06T11:22:01.612641+0000","last_clean":"2023-10-06T11:22:01.612641+0000","last_became_active":"2023-10-06T11:21:57.565455+0000","last_became_peered":"2023-10-06T11:21:57.565455+0000","last_unstale":"2023-10-06T11:22:01.612641+0000","last_undegraded":"2023-10-06T11:22:01.612641+0000","last_fullsized":"2023-10-06T11:22:01.612641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:28.522557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612331+0000","last_change":"2023-10-06T11:21:57.569901+0000","last_active":"2023-10-06T11:22:01.612331+0000","last_peered":"2023-10-06T11:22:01.612331+0000","last_clean":"2023-10-06T11:22:01.612331+0000","last_became_active":"2023-10-06T11:21:57.569642+0000","last_became_peered":"2023-10-06T11:21:57.569642+0000","last_unstale":"2023-10-06T11:22:01.612331+0000","last_undegraded":"2023-10-06T11:22:01.612331+0000","last_fullsized":"2023-10-06T11:22:01.612331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.950303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611118+0000","last_change":"2023-10-06T11:21:57.564154+0000","last_active":"2023-10-06T11:22:01.611118+0000","last_peered":"2023-10-06T11:22:01.611118+0000","last_clean":"2023-10-06T11:22:01.611118+0000","last_became_active":"2023-10-06T11:21:57.563840+0000","last_became_peered":"2023-10-06T11:21:57.563840+0000","last_unstale":"2023-10-06T11:22:01.611118+0000","last_undegraded":"2023-10-06T11:22:01.611118+0000","last_fullsized":"2023-10-06T11:22:01.611118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627637+0000","last_change":"2023-10-06T11:21:57.569223+0000","last_active":"2023-10-06T11:22:01.627637+0000","last_peered":"2023-10-06T11:22:01.627637+0000","last_clean":"2023-10-06T11:22:01.627637+0000","last_became_active":"2023-10-06T11:21:57.568967+0000","last_became_peered":"2023-10-06T11:21:57.568967+0000","last_unstale":"2023-10-06T11:22:01.627637+0000","last_undegraded":"2023-10-06T11:22:01.627637+0000","last_fullsized":"2023-10-06T11:22:01.627637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612585+0000","last_change":"2023-10-06T11:21:57.567578+0000","last_active":"2023-10-06T11:22:01.612585+0000","last_peered":"2023-10-06T11:22:01.612585+0000","last_clean":"2023-10-06T11:22:01.612585+0000","last_became_active":"2023-10-06T11:21:57.565977+0000","last_became_peered":"2023-10-06T11:21:57.565977+0000","last_unstale":"2023-10-06T11:22:01.612585+0000","last_undegraded":"2023-10-06T11:22:01.612585+0000","last_fullsized":"2023-10-06T11:22:01.612585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619766+0000","last_change":"2023-10-06T11:21:57.576090+0000","last_active":"2023-10-06T11:22:01.619766+0000","last_peered":"2023-10-06T11:22:01.619766+0000","last_clean":"2023-10-06T11:22:01.619766+0000","last_became_active":"2023-10-06T11:21:57.575667+0000","last_became_peered":"2023-10-06T11:21:57.575667+0000","last_unstale":"2023-10-06T11:22:01.619766+0000","last_undegraded":"2023-10-06T11:22:01.619766+0000","last_fullsized":"2023-10-06T11:22:01.619766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621178+0000","last_change":"2023-10-06T11:21:57.563777+0000","last_active":"2023-10-06T11:22:01.621178+0000","last_peered":"2023-10-06T11:22:01.621178+0000","last_clean":"2023-10-06T11:22:01.621178+0000","last_became_active":"2023-10-06T11:21:57.563485+0000","last_became_peered":"2023-10-06T11:21:57.563485+0000","last_unstale":"2023-10-06T11:22:01.621178+0000","last_undegraded":"2023-10-06T11:22:01.621178+0000","last_fullsized":"2023-10-06T11:22:01.621178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610375+0000","last_change":"2023-10-06T11:21:57.575519+0000","last_active":"2023-10-06T11:22:01.610375+0000","last_peered":"2023-10-06T11:22:01.610375+0000","last_clean":"2023-10-06T11:22:01.610375+0000","last_became_active":"2023-10-06T11:21:57.575122+0000","last_became_peered":"2023-10-06T11:21:57.575122+0000","last_unstale":"2023-10-06T11:22:01.610375+0000","last_undegraded":"2023-10-06T11:22:01.610375+0000","last_fullsized":"2023-10-06T11:22:01.610375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:01.582258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607465+0000","last_change":"2023-10-06T11:21:57.563806+0000","last_active":"2023-10-06T11:22:01.607465+0000","last_peered":"2023-10-06T11:22:01.607465+0000","last_clean":"2023-10-06T11:22:01.607465+0000","last_became_active":"2023-10-06T11:21:57.561796+0000","last_became_peered":"2023-10-06T11:21:57.561796+0000","last_unstale":"2023-10-06T11:22:01.607465+0000","last_undegraded":"2023-10-06T11:22:01.607465+0000","last_fullsized":"2023-10-06T11:22:01.607465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610200+0000","last_change":"2023-10-06T11:21:57.560661+0000","last_active":"2023-10-06T11:22:01.610200+0000","last_peered":"2023-10-06T11:22:01.610200+0000","last_clean":"2023-10-06T11:22:01.610200+0000","last_became_active":"2023-10-06T11:21:57.559848+0000","last_became_peered":"2023-10-06T11:21:57.559848+0000","last_unstale":"2023-10-06T11:22:01.610200+0000","last_undegraded":"2023-10-06T11:22:01.610200+0000","last_fullsized":"2023-10-06T11:22:01.610200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621548+0000","last_change":"2023-10-06T11:21:57.566423+0000","last_active":"2023-10-06T11:22:01.621548+0000","last_peered":"2023-10-06T11:22:01.621548+0000","last_clean":"2023-10-06T11:22:01.621548+0000","last_became_active":"2023-10-06T11:21:57.566064+0000","last_became_peered":"2023-10-06T11:21:57.566064+0000","last_unstale":"2023-10-06T11:22:01.621548+0000","last_undegraded":"2023-10-06T11:22:01.621548+0000","last_fullsized":"2023-10-06T11:22:01.621548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627537+0000","last_change":"2023-10-06T11:21:57.575695+0000","last_active":"2023-10-06T11:22:01.627537+0000","last_peered":"2023-10-06T11:22:01.627537+0000","last_clean":"2023-10-06T11:22:01.627537+0000","last_became_active":"2023-10-06T11:21:57.575418+0000","last_became_peered":"2023-10-06T11:21:57.575418+0000","last_unstale":"2023-10-06T11:22:01.627537+0000","last_undegraded":"2023-10-06T11:22:01.627537+0000","last_fullsized":"2023-10-06T11:22:01.627537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619861+0000","last_change":"2023-10-06T11:21:57.562165+0000","last_active":"2023-10-06T11:22:01.619861+0000","last_peered":"2023-10-06T11:22:01.619861+0000","last_clean":"2023-10-06T11:22:01.619861+0000","last_became_active":"2023-10-06T11:21:57.561878+0000","last_became_peered":"2023-10-06T11:21:57.561878+0000","last_unstale":"2023-10-06T11:22:01.619861+0000","last_undegraded":"2023-10-06T11:22:01.619861+0000","last_fullsized":"2023-10-06T11:22:01.619861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619664+0000","last_change":"2023-10-06T11:21:57.575950+0000","last_active":"2023-10-06T11:22:01.619664+0000","last_peered":"2023-10-06T11:22:01.619664+0000","last_clean":"2023-10-06T11:22:01.619664+0000","last_became_active":"2023-10-06T11:21:57.575280+0000","last_became_peered":"2023-10-06T11:21:57.575280+0000","last_unstale":"2023-10-06T11:22:01.619664+0000","last_undegraded":"2023-10-06T11:22:01.619664+0000","last_fullsized":"2023-10-06T11:22:01.619664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609933+0000","last_change":"2023-10-06T11:21:57.566684+0000","last_active":"2023-10-06T11:22:01.609933+0000","last_peered":"2023-10-06T11:22:01.609933+0000","last_clean":"2023-10-06T11:22:01.609933+0000","last_became_active":"2023-10-06T11:21:57.566358+0000","last_became_peered":"2023-10-06T11:21:57.566358+0000","last_unstale":"2023-10-06T11:22:01.609933+0000","last_undegraded":"2023-10-06T11:22:01.609933+0000","last_fullsized":"2023-10-06T11:22:01.609933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:50.324902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626719+0000","last_change":"2023-10-06T11:21:57.574099+0000","last_active":"2023-10-06T11:22:01.626719+0000","last_peered":"2023-10-06T11:22:01.626719+0000","last_clean":"2023-10-06T11:22:01.626719+0000","last_became_active":"2023-10-06T11:21:57.573592+0000","last_became_peered":"2023-10-06T11:21:57.573592+0000","last_unstale":"2023-10-06T11:22:01.626719+0000","last_undegraded":"2023-10-06T11:22:01.626719+0000","last_fullsized":"2023-10-06T11:22:01.626719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619959+0000","last_change":"2023-10-06T11:21:57.566310+0000","last_active":"2023-10-06T11:22:01.619959+0000","last_peered":"2023-10-06T11:22:01.619959+0000","last_clean":"2023-10-06T11:22:01.619959+0000","last_became_active":"2023-10-06T11:21:57.566014+0000","last_became_peered":"2023-10-06T11:21:57.566014+0000","last_unstale":"2023-10-06T11:22:01.619959+0000","last_undegraded":"2023-10-06T11:22:01.619959+0000","last_fullsized":"2023-10-06T11:22:01.619959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.268101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609857+0000","last_change":"2023-10-06T11:21:57.567982+0000","last_active":"2023-10-06T11:22:01.609857+0000","last_peered":"2023-10-06T11:22:01.609857+0000","last_clean":"2023-10-06T11:22:01.609857+0000","last_became_active":"2023-10-06T11:21:57.567667+0000","last_became_peered":"2023-10-06T11:21:57.567667+0000","last_unstale":"2023-10-06T11:22:01.609857+0000","last_undegraded":"2023-10-06T11:22:01.609857+0000","last_fullsized":"2023-10-06T11:22:01.609857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:14.276554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621434+0000","last_change":"2023-10-06T11:21:57.562657+0000","last_active":"2023-10-06T11:22:01.621434+0000","last_peered":"2023-10-06T11:22:01.621434+0000","last_clean":"2023-10-06T11:22:01.621434+0000","last_became_active":"2023-10-06T11:21:57.562338+0000","last_became_peered":"2023-10-06T11:21:57.562338+0000","last_unstale":"2023-10-06T11:22:01.621434+0000","last_undegraded":"2023-10-06T11:22:01.621434+0000","last_fullsized":"2023-10-06T11:22:01.621434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607967+0000","last_change":"2023-10-06T11:21:57.569451+0000","last_active":"2023-10-06T11:22:01.607967+0000","last_peered":"2023-10-06T11:22:01.607967+0000","last_clean":"2023-10-06T11:22:01.607967+0000","last_became_active":"2023-10-06T11:21:57.569124+0000","last_became_peered":"2023-10-06T11:21:57.569124+0000","last_unstale":"2023-10-06T11:22:01.607967+0000","last_undegraded":"2023-10-06T11:22:01.607967+0000","last_fullsized":"2023-10-06T11:22:01.607967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.833392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":703756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301652,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070188,"statfs":{"total":96636764160,"available":96327872512,"internally_reserved":0,"allocated":274432,"data_stored":110082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":286720,"data_stored":113228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":109548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302224,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069616,"statfs":{"total":96636764160,"available":96327286784,"internally_reserved":0,"allocated":860160,"data_stored":699826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":110012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:22:21.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:22:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.367+0000 7f6ca0c62700 1 Processor -- start 2023-10-06T11:22:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.367+0000 7f6ca0c62700 1 -- start start 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 0x7f6c9c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 0x7f6c9c15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9c15b3c0 con 0x7f6c9c15ab80 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c15f540 con 0x7f6c9c15af50 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c15e950 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c9c15ab80 0x7f6c9c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54950/0 (socket says 172.21.15.7:54950) 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/3064017450 learned_addr learned my addr 172.21.15.7:0/3064017450 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9affd700 1 --2- 172.21.15.7:0/3064017450 >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 0x7f6c9c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/3064017450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 0x7f6c9c15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074639776 0 0) 0x7f6c9c15b3c0 con 0x7f6c9c15ab80 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c7c003570 con 0x7f6c9c15ab80 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296272858 0 0) 0x7f6c7c003570 con 0x7f6c9c15ab80 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 -- 172.21.15.7:0/3064017450 >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 msgr2=0x7f6c9c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/3064017450 >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 0x7f6c9c0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 -- 172.21.15.7:0/3064017450 >> v1:172.21.15.7:6789/0 conn(0x7f6c9c15ab80 legacy=0x7f6c9c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 -- 172.21.15.7:0/3064017450 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0afd20 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9affd700 1 --2- 172.21.15.7:0/3064017450 >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 0x7f6c9c0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/3064017450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 0x7f6c9c15f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009e60 tx=0x7f6c94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d39d4c4cad1be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400e070 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400b7a0 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6c9a7fc700 1 -- 172.21.15.7:0/3064017450 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c001490 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- 172.21.15.7:0/3064017450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 msgr2=0x7f6c9c15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/3064017450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 0x7f6c9c15f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009e60 tx=0x7f6c94009780 comp rx=0 tx=0).stop 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- 172.21.15.7:0/3064017450 shutdown_connections 2023-10-06T11:22:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/3064017450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c0af8c0 0x7f6c9c15f9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/3064017450 >> v2:172.21.15.50:3300/0 conn(0x7f6c9c15af50 0x7f6c9c0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- 172.21.15.7:0/3064017450 >> 172.21.15.7:0/3064017450 conn(0x7f6c9c0688a0 msgr2=0x7f6c9c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- 172.21.15.7:0/3064017450 shutdown_connections 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.371+0000 7f6ca0c62700 1 -- 172.21.15.7:0/3064017450 wait complete. 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 Processor -- start 2023-10-06T11:22:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- start start 2023-10-06T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c9c15ab80 0x7f6c9c147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c15af50 0x7f6c9c147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0c96a0 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0ca490 con 0x7f6c9c15ab80 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0c9880 con 0x7f6c9c15af50 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41904/0 (socket says 172.21.15.7:41904) 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2163038876 learned_addr learned my addr 172.21.15.7:0/2163038876 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9affd700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c9c15ab80 0x7f6c9c147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c15af50 0x7f6c9c147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c15af50 msgr2=0x7f6c9c147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c15af50 0x7f6c9c147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c9c15ab80 msgr2=0x7f6c9c147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c9c15ab80 0x7f6c9c147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2163038876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009420 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00cb90 tx=0x7f6c8c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c007150 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0072d0 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c94009a20 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c007690 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.375+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c0c9e40 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.379+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c06c710 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.379+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6c8c004a40 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.379+0000 7f6c937fe700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 0x7f6c80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.379+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6c8c07c360 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.379+0000 7f6c9affd700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 0x7f6c80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.383+0000 7f6c9affd700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 0x7f6c80062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0c7e40 tx=0x7f6c840091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.387+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6c8c04c580 con 0x7f6c9c0af8c0 2023-10-06T11:22:21.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.539+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c9c0ca5d0 con 0x7f6c80060550 2023-10-06T11:22:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.559+0000 7f6c937fe700 1 -- 172.21.15.7:0/2163038876 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367520 (secure 0 0 0) 0x7f6c8401a030 con 0x7f6c80060550 2023-10-06T11:22:21.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 msgr2=0x7f6c80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 0x7f6c80062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0c7e40 tx=0x7f6c840091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 msgr2=0x7f6c9c156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00cb90 tx=0x7f6c8c001ac0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 shutdown_connections 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c80060550 0x7f6c80062a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c9c15af50 0x7f6c9c147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c9c15ab80 0x7f6c9c147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 --2- 172.21.15.7:0/2163038876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0af8c0 0x7f6c9c156c50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 >> 172.21.15.7:0/2163038876 conn(0x7f6c9c0688a0 msgr2=0x7f6c9c159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 shutdown_connections 2023-10-06T11:22:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.567+0000 7f6ca0c62700 1 -- 172.21.15.7:0/2163038876 wait complete. 2023-10-06T11:22:21.576 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:22:21.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-06T11:22:19.818682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414376,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752560344,"statfs":{"total":773094113280,"available":770621792256,"internally_reserved":0,"allocated":3383296,"data_stored":2065368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004789"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642766+0000","last_change":"2023-10-06T11:22:01.634221+0000","last_active":"2023-10-06T11:22:01.642766+0000","last_peered":"2023-10-06T11:22:01.642766+0000","last_clean":"2023-10-06T11:22:01.642766+0000","last_became_active":"2023-10-06T11:22:01.633727+0000","last_became_peered":"2023-10-06T11:22:01.633727+0000","last_unstale":"2023-10-06T11:22:01.642766+0000","last_undegraded":"2023-10-06T11:22:01.642766+0000","last_fullsized":"2023-10-06T11:22:01.642766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627705+0000","last_change":"2023-10-06T11:21:56.568541+0000","last_active":"2023-10-06T11:22:01.627705+0000","last_peered":"2023-10-06T11:22:01.627705+0000","last_clean":"2023-10-06T11:22:01.627705+0000","last_became_active":"2023-10-06T11:21:56.567104+0000","last_became_peered":"2023-10-06T11:21:56.567104+0000","last_unstale":"2023-10-06T11:22:01.627705+0000","last_undegraded":"2023-10-06T11:22:01.627705+0000","last_fullsized":"2023-10-06T11:22:01.627705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612705+0000","last_change":"2023-10-06T11:21:57.566064+0000","last_active":"2023-10-06T11:22:01.612705+0000","last_peered":"2023-10-06T11:22:01.612705+0000","last_clean":"2023-10-06T11:22:01.612705+0000","last_became_active":"2023-10-06T11:21:57.565589+0000","last_became_peered":"2023-10-06T11:21:57.565589+0000","last_unstale":"2023-10-06T11:22:01.612705+0000","last_undegraded":"2023-10-06T11:22:01.612705+0000","last_fullsized":"2023-10-06T11:22:01.612705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645194+0000","last_change":"2023-10-06T11:22:01.637184+0000","last_active":"2023-10-06T11:22:01.645194+0000","last_peered":"2023-10-06T11:22:01.645194+0000","last_clean":"2023-10-06T11:22:01.645194+0000","last_became_active":"2023-10-06T11:22:01.636773+0000","last_became_peered":"2023-10-06T11:22:01.636773+0000","last_unstale":"2023-10-06T11:22:01.645194+0000","last_undegraded":"2023-10-06T11:22:01.645194+0000","last_fullsized":"2023-10-06T11:22:01.645194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:41.238965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617318+0000","last_change":"2023-10-06T11:21:56.565259+0000","last_active":"2023-10-06T11:22:01.617318+0000","last_peered":"2023-10-06T11:22:01.617318+0000","last_clean":"2023-10-06T11:22:01.617318+0000","last_became_active":"2023-10-06T11:21:56.564927+0000","last_became_peered":"2023-10-06T11:21:56.564927+0000","last_unstale":"2023-10-06T11:22:01.617318+0000","last_undegraded":"2023-10-06T11:22:01.617318+0000","last_fullsized":"2023-10-06T11:22:01.617318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617577+0000","last_change":"2023-10-06T11:21:57.562278+0000","last_active":"2023-10-06T11:22:01.617577+0000","last_peered":"2023-10-06T11:22:01.617577+0000","last_clean":"2023-10-06T11:22:01.617577+0000","last_became_active":"2023-10-06T11:21:57.561972+0000","last_became_peered":"2023-10-06T11:21:57.561972+0000","last_unstale":"2023-10-06T11:22:01.617577+0000","last_undegraded":"2023-10-06T11:22:01.617577+0000","last_fullsized":"2023-10-06T11:22:01.617577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641228+0000","last_change":"2023-10-06T11:22:01.637805+0000","last_active":"2023-10-06T11:22:01.641228+0000","last_peered":"2023-10-06T11:22:01.641228+0000","last_clean":"2023-10-06T11:22:01.641228+0000","last_became_active":"2023-10-06T11:22:01.637404+0000","last_became_peered":"2023-10-06T11:22:01.637404+0000","last_unstale":"2023-10-06T11:22:01.641228+0000","last_undegraded":"2023-10-06T11:22:01.641228+0000","last_fullsized":"2023-10-06T11:22:01.641228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621633+0000","last_change":"2023-10-06T11:21:56.578707+0000","last_active":"2023-10-06T11:22:01.621633+0000","last_peered":"2023-10-06T11:22:01.621633+0000","last_clean":"2023-10-06T11:22:01.621633+0000","last_became_active":"2023-10-06T11:21:56.578411+0000","last_became_peered":"2023-10-06T11:21:56.578411+0000","last_unstale":"2023-10-06T11:22:01.621633+0000","last_undegraded":"2023-10-06T11:22:01.621633+0000","last_fullsized":"2023-10-06T11:22:01.621633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610166+0000","last_change":"2023-10-06T11:21:57.570591+0000","last_active":"2023-10-06T11:22:01.610166+0000","last_peered":"2023-10-06T11:22:01.610166+0000","last_clean":"2023-10-06T11:22:01.610166+0000","last_became_active":"2023-10-06T11:21:57.570166+0000","last_became_peered":"2023-10-06T11:21:57.570166+0000","last_unstale":"2023-10-06T11:22:01.610166+0000","last_undegraded":"2023-10-06T11:22:01.610166+0000","last_fullsized":"2023-10-06T11:22:01.610166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644696+0000","last_change":"2023-10-06T11:22:01.638015+0000","last_active":"2023-10-06T11:22:01.644696+0000","last_peered":"2023-10-06T11:22:01.644696+0000","last_clean":"2023-10-06T11:22:01.644696+0000","last_became_active":"2023-10-06T11:22:01.637477+0000","last_became_peered":"2023-10-06T11:22:01.637477+0000","last_unstale":"2023-10-06T11:22:01.644696+0000","last_undegraded":"2023-10-06T11:22:01.644696+0000","last_fullsized":"2023-10-06T11:22:01.644696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:56.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621240+0000","last_change":"2023-10-06T11:21:56.571616+0000","last_active":"2023-10-06T11:22:01.621240+0000","last_peered":"2023-10-06T11:22:01.621240+0000","last_clean":"2023-10-06T11:22:01.621240+0000","last_became_active":"2023-10-06T11:21:56.571289+0000","last_became_peered":"2023-10-06T11:21:56.571289+0000","last_unstale":"2023-10-06T11:22:01.621240+0000","last_undegraded":"2023-10-06T11:22:01.621240+0000","last_fullsized":"2023-10-06T11:22:01.621240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611658+0000","last_change":"2023-10-06T11:21:57.575085+0000","last_active":"2023-10-06T11:22:01.611658+0000","last_peered":"2023-10-06T11:22:01.611658+0000","last_clean":"2023-10-06T11:22:01.611658+0000","last_became_active":"2023-10-06T11:21:57.574716+0000","last_became_peered":"2023-10-06T11:21:57.574716+0000","last_unstale":"2023-10-06T11:22:01.611658+0000","last_undegraded":"2023-10-06T11:22:01.611658+0000","last_fullsized":"2023-10-06T11:22:01.611658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643969+0000","last_change":"2023-10-06T11:22:01.638673+0000","last_active":"2023-10-06T11:22:01.643969+0000","last_peered":"2023-10-06T11:22:01.643969+0000","last_clean":"2023-10-06T11:22:01.643969+0000","last_became_active":"2023-10-06T11:22:01.635630+0000","last_became_peered":"2023-10-06T11:22:01.635630+0000","last_unstale":"2023-10-06T11:22:01.643969+0000","last_undegraded":"2023-10-06T11:22:01.643969+0000","last_fullsized":"2023-10-06T11:22:01.643969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626800+0000","last_change":"2023-10-06T11:21:56.572349+0000","last_active":"2023-10-06T11:22:01.626800+0000","last_peered":"2023-10-06T11:22:01.626800+0000","last_clean":"2023-10-06T11:22:01.626800+0000","last_became_active":"2023-10-06T11:21:56.571848+0000","last_became_peered":"2023-10-06T11:21:56.571848+0000","last_unstale":"2023-10-06T11:22:01.626800+0000","last_undegraded":"2023-10-06T11:22:01.626800+0000","last_fullsized":"2023-10-06T11:22:01.626800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607694+0000","last_change":"2023-10-06T11:21:57.580595+0000","last_active":"2023-10-06T11:22:01.607694+0000","last_peered":"2023-10-06T11:22:01.607694+0000","last_clean":"2023-10-06T11:22:01.607694+0000","last_became_active":"2023-10-06T11:21:57.580286+0000","last_became_peered":"2023-10-06T11:21:57.580286+0000","last_unstale":"2023-10-06T11:22:01.607694+0000","last_undegraded":"2023-10-06T11:22:01.607694+0000","last_fullsized":"2023-10-06T11:22:01.607694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:04.725222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642661+0000","last_change":"2023-10-06T11:22:01.632283+0000","last_active":"2023-10-06T11:22:01.642661+0000","last_peered":"2023-10-06T11:22:01.642661+0000","last_clean":"2023-10-06T11:22:01.642661+0000","last_became_active":"2023-10-06T11:22:01.628222+0000","last_became_peered":"2023-10-06T11:22:01.628222+0000","last_unstale":"2023-10-06T11:22:01.642661+0000","last_undegraded":"2023-10-06T11:22:01.642661+0000","last_fullsized":"2023-10-06T11:22:01.642661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:32.168166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610453+0000","last_change":"2023-10-06T11:21:56.571031+0000","last_active":"2023-10-06T11:22:01.610453+0000","last_peered":"2023-10-06T11:22:01.610453+0000","last_clean":"2023-10-06T11:22:01.610453+0000","last_became_active":"2023-10-06T11:21:56.570724+0000","last_became_peered":"2023-10-06T11:21:56.570724+0000","last_unstale":"2023-10-06T11:22:01.610453+0000","last_undegraded":"2023-10-06T11:22:01.610453+0000","last_fullsized":"2023-10-06T11:22:01.610453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616088+0000","last_change":"2023-10-06T11:21:57.562757+0000","last_active":"2023-10-06T11:22:01.616088+0000","last_peered":"2023-10-06T11:22:01.616088+0000","last_clean":"2023-10-06T11:22:01.616088+0000","last_became_active":"2023-10-06T11:21:57.562240+0000","last_became_peered":"2023-10-06T11:21:57.562240+0000","last_unstale":"2023-10-06T11:22:01.616088+0000","last_undegraded":"2023-10-06T11:22:01.616088+0000","last_fullsized":"2023-10-06T11:22:01.616088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.636503+0000","last_change":"2023-10-06T11:22:01.635655+0000","last_active":"2023-10-06T11:22:01.636503+0000","last_peered":"2023-10-06T11:22:01.636503+0000","last_clean":"2023-10-06T11:22:01.636503+0000","last_became_active":"2023-10-06T11:22:01.635167+0000","last_became_peered":"2023-10-06T11:22:01.635167+0000","last_unstale":"2023-10-06T11:22:01.636503+0000","last_undegraded":"2023-10-06T11:22:01.636503+0000","last_fullsized":"2023-10-06T11:22:01.636503+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:16.080135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617232+0000","last_change":"2023-10-06T11:21:56.569415+0000","last_active":"2023-10-06T11:22:01.617232+0000","last_peered":"2023-10-06T11:22:01.617232+0000","last_clean":"2023-10-06T11:22:01.617232+0000","last_became_active":"2023-10-06T11:21:56.569132+0000","last_became_peered":"2023-10-06T11:21:56.569132+0000","last_unstale":"2023-10-06T11:22:01.617232+0000","last_undegraded":"2023-10-06T11:22:01.617232+0000","last_fullsized":"2023-10-06T11:22:01.617232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627493+0000","last_change":"2023-10-06T11:21:57.566844+0000","last_active":"2023-10-06T11:22:01.627493+0000","last_peered":"2023-10-06T11:22:01.627493+0000","last_clean":"2023-10-06T11:22:01.627493+0000","last_became_active":"2023-10-06T11:21:57.566564+0000","last_became_peered":"2023-10-06T11:21:57.566564+0000","last_unstale":"2023-10-06T11:22:01.627493+0000","last_undegraded":"2023-10-06T11:22:01.627493+0000","last_fullsized":"2023-10-06T11:22:01.627493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640584+0000","last_change":"2023-10-06T11:22:01.639744+0000","last_active":"2023-10-06T11:22:01.640584+0000","last_peered":"2023-10-06T11:22:01.640584+0000","last_clean":"2023-10-06T11:22:01.640584+0000","last_became_active":"2023-10-06T11:22:01.639253+0000","last_became_peered":"2023-10-06T11:22:01.639253+0000","last_unstale":"2023-10-06T11:22:01.640584+0000","last_undegraded":"2023-10-06T11:22:01.640584+0000","last_fullsized":"2023-10-06T11:22:01.640584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:08.811821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615560+0000","last_change":"2023-10-06T11:21:56.574525+0000","last_active":"2023-10-06T11:22:01.615560+0000","last_peered":"2023-10-06T11:22:01.615560+0000","last_clean":"2023-10-06T11:22:01.615560+0000","last_became_active":"2023-10-06T11:21:56.574182+0000","last_became_peered":"2023-10-06T11:21:56.574182+0000","last_unstale":"2023-10-06T11:22:01.615560+0000","last_undegraded":"2023-10-06T11:22:01.615560+0000","last_fullsized":"2023-10-06T11:22:01.615560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620100+0000","last_change":"2023-10-06T11:21:57.564759+0000","last_active":"2023-10-06T11:22:01.620100+0000","last_peered":"2023-10-06T11:22:01.620100+0000","last_clean":"2023-10-06T11:22:01.620100+0000","last_became_active":"2023-10-06T11:21:57.564385+0000","last_became_peered":"2023-10-06T11:21:57.564385+0000","last_unstale":"2023-10-06T11:22:01.620100+0000","last_undegraded":"2023-10-06T11:22:01.620100+0000","last_fullsized":"2023-10-06T11:22:01.620100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615364+0000","last_change":"2023-10-06T11:21:57.567825+0000","last_active":"2023-10-06T11:22:01.615364+0000","last_peered":"2023-10-06T11:22:01.615364+0000","last_clean":"2023-10-06T11:22:01.615364+0000","last_became_active":"2023-10-06T11:21:57.562342+0000","last_became_peered":"2023-10-06T11:21:57.562342+0000","last_unstale":"2023-10-06T11:22:01.615364+0000","last_undegraded":"2023-10-06T11:22:01.615364+0000","last_fullsized":"2023-10-06T11:22:01.615364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627753+0000","last_change":"2023-10-06T11:21:56.573919+0000","last_active":"2023-10-06T11:22:01.627753+0000","last_peered":"2023-10-06T11:22:01.627753+0000","last_clean":"2023-10-06T11:22:01.627753+0000","last_became_active":"2023-10-06T11:21:56.573605+0000","last_became_peered":"2023-10-06T11:21:56.573605+0000","last_unstale":"2023-10-06T11:22:01.627753+0000","last_undegraded":"2023-10-06T11:22:01.627753+0000","last_fullsized":"2023-10-06T11:22:01.627753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639931+0000","last_change":"2023-10-06T11:22:01.637440+0000","last_active":"2023-10-06T11:22:01.639931+0000","last_peered":"2023-10-06T11:22:01.639931+0000","last_clean":"2023-10-06T11:22:01.639931+0000","last_became_active":"2023-10-06T11:22:01.637003+0000","last_became_peered":"2023-10-06T11:22:01.637003+0000","last_unstale":"2023-10-06T11:22:01.639931+0000","last_undegraded":"2023-10-06T11:22:01.639931+0000","last_fullsized":"2023-10-06T11:22:01.639931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.578282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617665+0000","last_change":"2023-10-06T11:21:57.572470+0000","last_active":"2023-10-06T11:22:01.617665+0000","last_peered":"2023-10-06T11:22:01.617665+0000","last_clean":"2023-10-06T11:22:01.617665+0000","last_became_active":"2023-10-06T11:21:57.568699+0000","last_became_peered":"2023-10-06T11:21:57.568699+0000","last_unstale":"2023-10-06T11:22:01.617665+0000","last_undegraded":"2023-10-06T11:22:01.617665+0000","last_fullsized":"2023-10-06T11:22:01.617665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621640+0000","last_change":"2023-10-06T11:21:56.576266+0000","last_active":"2023-10-06T11:22:01.621640+0000","last_peered":"2023-10-06T11:22:01.621640+0000","last_clean":"2023-10-06T11:22:01.621640+0000","last_became_active":"2023-10-06T11:21:56.575879+0000","last_became_peered":"2023-10-06T11:21:56.575879+0000","last_unstale":"2023-10-06T11:22:01.621640+0000","last_undegraded":"2023-10-06T11:22:01.621640+0000","last_fullsized":"2023-10-06T11:22:01.621640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.645265+0000","last_change":"2023-10-06T11:22:01.642796+0000","last_active":"2023-10-06T11:22:01.645265+0000","last_peered":"2023-10-06T11:22:01.645265+0000","last_clean":"2023-10-06T11:22:01.645265+0000","last_became_active":"2023-10-06T11:22:01.642102+0000","last_became_peered":"2023-10-06T11:22:01.642102+0000","last_unstale":"2023-10-06T11:22:01.645265+0000","last_undegraded":"2023-10-06T11:22:01.645265+0000","last_fullsized":"2023-10-06T11:22:01.645265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620311+0000","last_change":"2023-10-06T11:21:56.577336+0000","last_active":"2023-10-06T11:22:01.620311+0000","last_peered":"2023-10-06T11:22:01.620311+0000","last_clean":"2023-10-06T11:22:01.620311+0000","last_became_active":"2023-10-06T11:21:56.574934+0000","last_became_peered":"2023-10-06T11:21:56.574934+0000","last_unstale":"2023-10-06T11:22:01.620311+0000","last_undegraded":"2023-10-06T11:22:01.620311+0000","last_fullsized":"2023-10-06T11:22:01.620311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:58.978806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615123+0000","last_change":"2023-10-06T11:21:57.565736+0000","last_active":"2023-10-06T11:22:01.615123+0000","last_peered":"2023-10-06T11:22:01.615123+0000","last_clean":"2023-10-06T11:22:01.615123+0000","last_became_active":"2023-10-06T11:21:57.565078+0000","last_became_peered":"2023-10-06T11:21:57.565078+0000","last_unstale":"2023-10-06T11:22:01.615123+0000","last_undegraded":"2023-10-06T11:22:01.615123+0000","last_fullsized":"2023-10-06T11:22:01.615123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637186+0000","last_change":"2023-10-06T11:22:01.635715+0000","last_active":"2023-10-06T11:22:01.637186+0000","last_peered":"2023-10-06T11:22:01.637186+0000","last_clean":"2023-10-06T11:22:01.637186+0000","last_became_active":"2023-10-06T11:22:01.635213+0000","last_became_peered":"2023-10-06T11:22:01.635213+0000","last_unstale":"2023-10-06T11:22:01.637186+0000","last_undegraded":"2023-10-06T11:22:01.637186+0000","last_fullsized":"2023-10-06T11:22:01.637186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:00.176987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-06T11:22:07.460343+0000","last_change":"2023-10-06T11:21:56.573406+0000","last_active":"2023-10-06T11:22:07.460343+0000","last_peered":"2023-10-06T11:22:07.460343+0000","last_clean":"2023-10-06T11:22:07.460343+0000","last_became_active":"2023-10-06T11:21:56.572858+0000","last_became_peered":"2023-10-06T11:21:56.572858+0000","last_unstale":"2023-10-06T11:22:07.460343+0000","last_undegraded":"2023-10-06T11:22:07.460343+0000","last_fullsized":"2023-10-06T11:22:07.460343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611768+0000","last_change":"2023-10-06T11:21:57.566510+0000","last_active":"2023-10-06T11:22:01.611768+0000","last_peered":"2023-10-06T11:22:01.611768+0000","last_clean":"2023-10-06T11:22:01.611768+0000","last_became_active":"2023-10-06T11:21:57.566168+0000","last_became_peered":"2023-10-06T11:21:57.566168+0000","last_unstale":"2023-10-06T11:22:01.611768+0000","last_undegraded":"2023-10-06T11:22:01.611768+0000","last_fullsized":"2023-10-06T11:22:01.611768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:03.939980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644970+0000","last_change":"2023-10-06T11:22:01.640707+0000","last_active":"2023-10-06T11:22:01.644970+0000","last_peered":"2023-10-06T11:22:01.644970+0000","last_clean":"2023-10-06T11:22:01.644970+0000","last_became_active":"2023-10-06T11:22:01.640294+0000","last_became_peered":"2023-10-06T11:22:01.640294+0000","last_unstale":"2023-10-06T11:22:01.644970+0000","last_undegraded":"2023-10-06T11:22:01.644970+0000","last_fullsized":"2023-10-06T11:22:01.644970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:07.995705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615237+0000","last_change":"2023-10-06T11:21:57.565357+0000","last_active":"2023-10-06T11:22:01.615237+0000","last_peered":"2023-10-06T11:22:01.615237+0000","last_clean":"2023-10-06T11:22:01.615237+0000","last_became_active":"2023-10-06T11:21:57.560887+0000","last_became_peered":"2023-10-06T11:21:57.560887+0000","last_unstale":"2023-10-06T11:22:01.615237+0000","last_undegraded":"2023-10-06T11:22:01.615237+0000","last_fullsized":"2023-10-06T11:22:01.615237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620136+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:22:01.620136+0000","last_peered":"2023-10-06T11:22:01.620136+0000","last_clean":"2023-10-06T11:22:01.620136+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:22:01.620136+0000","last_undegraded":"2023-10-06T11:22:01.620136+0000","last_fullsized":"2023-10-06T11:22:01.620136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621725+0000","last_change":"2023-10-06T11:21:56.569726+0000","last_active":"2023-10-06T11:22:01.621725+0000","last_peered":"2023-10-06T11:22:01.621725+0000","last_clean":"2023-10-06T11:22:01.621725+0000","last_became_active":"2023-10-06T11:21:56.569394+0000","last_became_peered":"2023-10-06T11:21:56.569394+0000","last_unstale":"2023-10-06T11:22:01.621725+0000","last_undegraded":"2023-10-06T11:22:01.621725+0000","last_fullsized":"2023-10-06T11:22:01.621725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.640548+0000","last_change":"2023-10-06T11:22:01.637613+0000","last_active":"2023-10-06T11:22:01.640548+0000","last_peered":"2023-10-06T11:22:01.640548+0000","last_clean":"2023-10-06T11:22:01.640548+0000","last_became_active":"2023-10-06T11:22:01.636818+0000","last_became_peered":"2023-10-06T11:22:01.636818+0000","last_unstale":"2023-10-06T11:22:01.640548+0000","last_undegraded":"2023-10-06T11:22:01.640548+0000","last_fullsized":"2023-10-06T11:22:01.640548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:42.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626583+0000","last_change":"2023-10-06T11:21:57.556132+0000","last_active":"2023-10-06T11:22:01.626583+0000","last_peered":"2023-10-06T11:22:01.626583+0000","last_clean":"2023-10-06T11:22:01.626583+0000","last_became_active":"2023-10-06T11:21:57.555766+0000","last_became_peered":"2023-10-06T11:21:57.555766+0000","last_unstale":"2023-10-06T11:22:01.626583+0000","last_undegraded":"2023-10-06T11:22:01.626583+0000","last_fullsized":"2023-10-06T11:22:01.626583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620468+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:22:01.620468+0000","last_peered":"2023-10-06T11:22:01.620468+0000","last_clean":"2023-10-06T11:22:01.620468+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:22:01.620468+0000","last_undegraded":"2023-10-06T11:22:01.620468+0000","last_fullsized":"2023-10-06T11:22:01.620468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619168+0000","last_change":"2023-10-06T11:21:56.570891+0000","last_active":"2023-10-06T11:22:01.619168+0000","last_peered":"2023-10-06T11:22:01.619168+0000","last_clean":"2023-10-06T11:22:01.619168+0000","last_became_active":"2023-10-06T11:21:56.570569+0000","last_became_peered":"2023-10-06T11:21:56.570569+0000","last_unstale":"2023-10-06T11:22:01.619168+0000","last_undegraded":"2023-10-06T11:22:01.619168+0000","last_fullsized":"2023-10-06T11:22:01.619168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634770+0000","last_change":"2023-10-06T11:22:01.630420+0000","last_active":"2023-10-06T11:22:01.634770+0000","last_peered":"2023-10-06T11:22:01.634770+0000","last_clean":"2023-10-06T11:22:01.634770+0000","last_became_active":"2023-10-06T11:22:01.629842+0000","last_became_peered":"2023-10-06T11:22:01.629842+0000","last_unstale":"2023-10-06T11:22:01.634770+0000","last_undegraded":"2023-10-06T11:22:01.634770+0000","last_fullsized":"2023-10-06T11:22:01.634770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:49.797072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615458+0000","last_change":"2023-10-06T11:21:57.568394+0000","last_active":"2023-10-06T11:22:01.615458+0000","last_peered":"2023-10-06T11:22:01.615458+0000","last_clean":"2023-10-06T11:22:01.615458+0000","last_became_active":"2023-10-06T11:21:57.568120+0000","last_became_peered":"2023-10-06T11:21:57.568120+0000","last_unstale":"2023-10-06T11:22:01.615458+0000","last_undegraded":"2023-10-06T11:22:01.615458+0000","last_fullsized":"2023-10-06T11:22:01.615458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:45.741347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610919+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:22:01.610919+0000","last_peered":"2023-10-06T11:22:01.610919+0000","last_clean":"2023-10-06T11:22:01.610919+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:22:01.610919+0000","last_undegraded":"2023-10-06T11:22:01.610919+0000","last_fullsized":"2023-10-06T11:22:01.610919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616279+0000","last_change":"2023-10-06T11:21:56.574787+0000","last_active":"2023-10-06T11:22:01.616279+0000","last_peered":"2023-10-06T11:22:01.616279+0000","last_clean":"2023-10-06T11:22:01.616279+0000","last_became_active":"2023-10-06T11:21:56.574471+0000","last_became_peered":"2023-10-06T11:21:56.574471+0000","last_unstale":"2023-10-06T11:22:01.616279+0000","last_undegraded":"2023-10-06T11:22:01.616279+0000","last_fullsized":"2023-10-06T11:22:01.616279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642714+0000","last_change":"2023-10-06T11:22:01.632835+0000","last_active":"2023-10-06T11:22:01.642714+0000","last_peered":"2023-10-06T11:22:01.642714+0000","last_clean":"2023-10-06T11:22:01.642714+0000","last_became_active":"2023-10-06T11:22:01.632391+0000","last_became_peered":"2023-10-06T11:22:01.632391+0000","last_unstale":"2023-10-06T11:22:01.642714+0000","last_undegraded":"2023-10-06T11:22:01.642714+0000","last_fullsized":"2023-10-06T11:22:01.642714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:08.263854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611307+0000","last_change":"2023-10-06T11:21:57.561918+0000","last_active":"2023-10-06T11:22:01.611307+0000","last_peered":"2023-10-06T11:22:01.611307+0000","last_clean":"2023-10-06T11:22:01.611307+0000","last_became_active":"2023-10-06T11:21:57.560917+0000","last_became_peered":"2023-10-06T11:21:57.560917+0000","last_unstale":"2023-10-06T11:22:01.611307+0000","last_undegraded":"2023-10-06T11:22:01.611307+0000","last_fullsized":"2023-10-06T11:22:01.611307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616596+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:22:01.616596+0000","last_peered":"2023-10-06T11:22:01.616596+0000","last_clean":"2023-10-06T11:22:01.616596+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:22:01.616596+0000","last_undegraded":"2023-10-06T11:22:01.616596+0000","last_fullsized":"2023-10-06T11:22:01.616596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:02.262049+0000","last_change":"2023-10-06T11:21:56.573826+0000","last_active":"2023-10-06T11:22:02.262049+0000","last_peered":"2023-10-06T11:22:02.262049+0000","last_clean":"2023-10-06T11:22:02.262049+0000","last_became_active":"2023-10-06T11:21:56.573509+0000","last_became_peered":"2023-10-06T11:21:56.573509+0000","last_unstale":"2023-10-06T11:22:02.262049+0000","last_undegraded":"2023-10-06T11:22:02.262049+0000","last_fullsized":"2023-10-06T11:22:02.262049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634403+0000","last_change":"2023-10-06T11:22:01.631312+0000","last_active":"2023-10-06T11:22:01.634403+0000","last_peered":"2023-10-06T11:22:01.634403+0000","last_clean":"2023-10-06T11:22:01.634403+0000","last_became_active":"2023-10-06T11:22:01.630877+0000","last_became_peered":"2023-10-06T11:22:01.630877+0000","last_unstale":"2023-10-06T11:22:01.634403+0000","last_undegraded":"2023-10-06T11:22:01.634403+0000","last_fullsized":"2023-10-06T11:22:01.634403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:03:28.590751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627610+0000","last_change":"2023-10-06T11:21:57.567907+0000","last_active":"2023-10-06T11:22:01.627610+0000","last_peered":"2023-10-06T11:22:01.627610+0000","last_clean":"2023-10-06T11:22:01.627610+0000","last_became_active":"2023-10-06T11:21:57.567544+0000","last_became_peered":"2023-10-06T11:21:57.567544+0000","last_unstale":"2023-10-06T11:22:01.627610+0000","last_undegraded":"2023-10-06T11:22:01.627610+0000","last_fullsized":"2023-10-06T11:22:01.627610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.625024+0000","last_change":"2023-10-06T11:21:56.571499+0000","last_active":"2023-10-06T11:22:01.625024+0000","last_peered":"2023-10-06T11:22:01.625024+0000","last_clean":"2023-10-06T11:22:01.625024+0000","last_became_active":"2023-10-06T11:21:56.571176+0000","last_became_peered":"2023-10-06T11:21:56.571176+0000","last_unstale":"2023-10-06T11:22:01.625024+0000","last_undegraded":"2023-10-06T11:22:01.625024+0000","last_fullsized":"2023-10-06T11:22:01.625024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:44:16.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616847+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:22:01.616847+0000","last_peered":"2023-10-06T11:22:01.616847+0000","last_clean":"2023-10-06T11:22:01.616847+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:22:01.616847+0000","last_undegraded":"2023-10-06T11:22:01.616847+0000","last_fullsized":"2023-10-06T11:22:01.616847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614070+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:22:01.614070+0000","last_peered":"2023-10-06T11:22:01.614070+0000","last_clean":"2023-10-06T11:22:01.614070+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:22:01.614070+0000","last_undegraded":"2023-10-06T11:22:01.614070+0000","last_fullsized":"2023-10-06T11:22:01.614070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635892+0000","last_change":"2023-10-06T11:22:01.632615+0000","last_active":"2023-10-06T11:22:01.635892+0000","last_peered":"2023-10-06T11:22:01.635892+0000","last_clean":"2023-10-06T11:22:01.635892+0000","last_became_active":"2023-10-06T11:22:01.632167+0000","last_became_peered":"2023-10-06T11:22:01.632167+0000","last_unstale":"2023-10-06T11:22:01.635892+0000","last_undegraded":"2023-10-06T11:22:01.635892+0000","last_fullsized":"2023-10-06T11:22:01.635892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:35.069167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614553+0000","last_change":"2023-10-06T11:21:57.558911+0000","last_active":"2023-10-06T11:22:01.614553+0000","last_peered":"2023-10-06T11:22:01.614553+0000","last_clean":"2023-10-06T11:22:01.614553+0000","last_became_active":"2023-10-06T11:21:57.555525+0000","last_became_peered":"2023-10-06T11:21:57.555525+0000","last_unstale":"2023-10-06T11:22:01.614553+0000","last_undegraded":"2023-10-06T11:22:01.614553+0000","last_fullsized":"2023-10-06T11:22:01.614553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620129+0000","last_change":"2023-10-06T11:21:56.566441+0000","last_active":"2023-10-06T11:22:01.620129+0000","last_peered":"2023-10-06T11:22:01.620129+0000","last_clean":"2023-10-06T11:22:01.620129+0000","last_became_active":"2023-10-06T11:21:56.566070+0000","last_became_peered":"2023-10-06T11:21:56.566070+0000","last_unstale":"2023-10-06T11:22:01.620129+0000","last_undegraded":"2023-10-06T11:22:01.620129+0000","last_fullsized":"2023-10-06T11:22:01.620129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616748+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:22:01.616748+0000","last_peered":"2023-10-06T11:22:01.616748+0000","last_clean":"2023-10-06T11:22:01.616748+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:22:01.616748+0000","last_undegraded":"2023-10-06T11:22:01.616748+0000","last_fullsized":"2023-10-06T11:22:01.616748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637816+0000","last_change":"2023-10-06T11:22:01.632881+0000","last_active":"2023-10-06T11:22:01.637816+0000","last_peered":"2023-10-06T11:22:01.637816+0000","last_clean":"2023-10-06T11:22:01.637816+0000","last_became_active":"2023-10-06T11:22:01.632330+0000","last_became_peered":"2023-10-06T11:22:01.632330+0000","last_unstale":"2023-10-06T11:22:01.637816+0000","last_undegraded":"2023-10-06T11:22:01.637816+0000","last_fullsized":"2023-10-06T11:22:01.637816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:38.278034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619088+0000","last_change":"2023-10-06T11:21:57.565393+0000","last_active":"2023-10-06T11:22:01.619088+0000","last_peered":"2023-10-06T11:22:01.619088+0000","last_clean":"2023-10-06T11:22:01.619088+0000","last_became_active":"2023-10-06T11:21:57.565128+0000","last_became_peered":"2023-10-06T11:21:57.565128+0000","last_unstale":"2023-10-06T11:22:01.619088+0000","last_undegraded":"2023-10-06T11:22:01.619088+0000","last_fullsized":"2023-10-06T11:22:01.619088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611906+0000","last_change":"2023-10-06T11:21:56.573109+0000","last_active":"2023-10-06T11:22:01.611906+0000","last_peered":"2023-10-06T11:22:01.611906+0000","last_clean":"2023-10-06T11:22:01.611906+0000","last_became_active":"2023-10-06T11:21:56.572783+0000","last_became_peered":"2023-10-06T11:21:56.572783+0000","last_unstale":"2023-10-06T11:22:01.611906+0000","last_undegraded":"2023-10-06T11:22:01.611906+0000","last_fullsized":"2023-10-06T11:22:01.611906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620654+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:22:01.620654+0000","last_peered":"2023-10-06T11:22:01.620654+0000","last_clean":"2023-10-06T11:22:01.620654+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:22:01.620654+0000","last_undegraded":"2023-10-06T11:22:01.620654+0000","last_fullsized":"2023-10-06T11:22:01.620654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643525+0000","last_change":"2023-10-06T11:22:01.641973+0000","last_active":"2023-10-06T11:22:01.643525+0000","last_peered":"2023-10-06T11:22:01.643525+0000","last_clean":"2023-10-06T11:22:01.643525+0000","last_became_active":"2023-10-06T11:22:01.641394+0000","last_became_peered":"2023-10-06T11:22:01.641394+0000","last_unstale":"2023-10-06T11:22:01.643525+0000","last_undegraded":"2023-10-06T11:22:01.643525+0000","last_fullsized":"2023-10-06T11:22:01.643525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620183+0000","last_change":"2023-10-06T11:21:57.564937+0000","last_active":"2023-10-06T11:22:01.620183+0000","last_peered":"2023-10-06T11:22:01.620183+0000","last_clean":"2023-10-06T11:22:01.620183+0000","last_became_active":"2023-10-06T11:21:57.564603+0000","last_became_peered":"2023-10-06T11:21:57.564603+0000","last_unstale":"2023-10-06T11:22:01.620183+0000","last_undegraded":"2023-10-06T11:22:01.620183+0000","last_fullsized":"2023-10-06T11:22:01.620183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615864+0000","last_change":"2023-10-06T11:21:56.567945+0000","last_active":"2023-10-06T11:22:01.615864+0000","last_peered":"2023-10-06T11:22:01.615864+0000","last_clean":"2023-10-06T11:22:01.615864+0000","last_became_active":"2023-10-06T11:21:56.567592+0000","last_became_peered":"2023-10-06T11:21:56.567592+0000","last_unstale":"2023-10-06T11:22:01.615864+0000","last_undegraded":"2023-10-06T11:22:01.615864+0000","last_fullsized":"2023-10-06T11:22:01.615864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:00.577535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616944+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:22:01.616944+0000","last_peered":"2023-10-06T11:22:01.616944+0000","last_clean":"2023-10-06T11:22:01.616944+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:22:01.616944+0000","last_undegraded":"2023-10-06T11:22:01.616944+0000","last_fullsized":"2023-10-06T11:22:01.616944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.644814+0000","last_change":"2023-10-06T11:22:01.639193+0000","last_active":"2023-10-06T11:22:01.644814+0000","last_peered":"2023-10-06T11:22:01.644814+0000","last_clean":"2023-10-06T11:22:01.644814+0000","last_became_active":"2023-10-06T11:22:01.638730+0000","last_became_peered":"2023-10-06T11:22:01.638730+0000","last_unstale":"2023-10-06T11:22:01.644814+0000","last_undegraded":"2023-10-06T11:22:01.644814+0000","last_fullsized":"2023-10-06T11:22:01.644814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614448+0000","last_change":"2023-10-06T11:21:57.565173+0000","last_active":"2023-10-06T11:22:01.614448+0000","last_peered":"2023-10-06T11:22:01.614448+0000","last_clean":"2023-10-06T11:22:01.614448+0000","last_became_active":"2023-10-06T11:21:57.564865+0000","last_became_peered":"2023-10-06T11:21:57.564865+0000","last_unstale":"2023-10-06T11:22:01.614448+0000","last_undegraded":"2023-10-06T11:22:01.614448+0000","last_fullsized":"2023-10-06T11:22:01.614448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616901+0000","last_change":"2023-10-06T11:21:56.570015+0000","last_active":"2023-10-06T11:22:01.616901+0000","last_peered":"2023-10-06T11:22:01.616901+0000","last_clean":"2023-10-06T11:22:01.616901+0000","last_became_active":"2023-10-06T11:21:56.569530+0000","last_became_peered":"2023-10-06T11:21:56.569530+0000","last_unstale":"2023-10-06T11:22:01.616901+0000","last_undegraded":"2023-10-06T11:22:01.616901+0000","last_fullsized":"2023-10-06T11:22:01.616901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.642672+0000","last_change":"2023-10-06T11:22:01.632016+0000","last_active":"2023-10-06T11:22:01.642672+0000","last_peered":"2023-10-06T11:22:01.642672+0000","last_clean":"2023-10-06T11:22:01.642672+0000","last_became_active":"2023-10-06T11:22:01.629544+0000","last_became_peered":"2023-10-06T11:22:01.629544+0000","last_unstale":"2023-10-06T11:22:01.642672+0000","last_undegraded":"2023-10-06T11:22:01.642672+0000","last_fullsized":"2023-10-06T11:22:01.642672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:20.535717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611336+0000","last_change":"2023-10-06T11:21:57.566827+0000","last_active":"2023-10-06T11:22:01.611336+0000","last_peered":"2023-10-06T11:22:01.611336+0000","last_clean":"2023-10-06T11:22:01.611336+0000","last_became_active":"2023-10-06T11:21:57.566463+0000","last_became_peered":"2023-10-06T11:21:57.566463+0000","last_unstale":"2023-10-06T11:22:01.611336+0000","last_undegraded":"2023-10-06T11:22:01.611336+0000","last_fullsized":"2023-10-06T11:22:01.611336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611609+0000","last_change":"2023-10-06T11:21:56.573082+0000","last_active":"2023-10-06T11:22:01.611609+0000","last_peered":"2023-10-06T11:22:01.611609+0000","last_clean":"2023-10-06T11:22:01.611609+0000","last_became_active":"2023-10-06T11:21:56.572363+0000","last_became_peered":"2023-10-06T11:21:56.572363+0000","last_unstale":"2023-10-06T11:22:01.611609+0000","last_undegraded":"2023-10-06T11:22:01.611609+0000","last_fullsized":"2023-10-06T11:22:01.611609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.628822+0000","last_change":"2023-10-06T11:22:01.626277+0000","last_active":"2023-10-06T11:22:01.628822+0000","last_peered":"2023-10-06T11:22:01.628822+0000","last_clean":"2023-10-06T11:22:01.628822+0000","last_became_active":"2023-10-06T11:22:01.625806+0000","last_became_peered":"2023-10-06T11:22:01.625806+0000","last_unstale":"2023-10-06T11:22:01.628822+0000","last_undegraded":"2023-10-06T11:22:01.628822+0000","last_fullsized":"2023-10-06T11:22:01.628822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611575+0000","last_change":"2023-10-06T11:21:57.564883+0000","last_active":"2023-10-06T11:22:01.611575+0000","last_peered":"2023-10-06T11:22:01.611575+0000","last_clean":"2023-10-06T11:22:01.611575+0000","last_became_active":"2023-10-06T11:21:57.564586+0000","last_became_peered":"2023-10-06T11:21:57.564586+0000","last_unstale":"2023-10-06T11:22:01.611575+0000","last_undegraded":"2023-10-06T11:22:01.611575+0000","last_fullsized":"2023-10-06T11:22:01.611575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:39:12.024410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621041+0000","last_change":"2023-10-06T11:21:56.576790+0000","last_active":"2023-10-06T11:22:01.621041+0000","last_peered":"2023-10-06T11:22:01.621041+0000","last_clean":"2023-10-06T11:22:01.621041+0000","last_became_active":"2023-10-06T11:21:56.576463+0000","last_became_peered":"2023-10-06T11:21:56.576463+0000","last_unstale":"2023-10-06T11:22:01.621041+0000","last_undegraded":"2023-10-06T11:22:01.621041+0000","last_fullsized":"2023-10-06T11:22:01.621041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643771+0000","last_change":"2023-10-06T11:22:01.639619+0000","last_active":"2023-10-06T11:22:01.643771+0000","last_peered":"2023-10-06T11:22:01.643771+0000","last_clean":"2023-10-06T11:22:01.643771+0000","last_became_active":"2023-10-06T11:22:01.638615+0000","last_became_peered":"2023-10-06T11:22:01.638615+0000","last_unstale":"2023-10-06T11:22:01.643771+0000","last_undegraded":"2023-10-06T11:22:01.643771+0000","last_fullsized":"2023-10-06T11:22:01.643771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:00.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612709+0000","last_change":"2023-10-06T11:21:57.577150+0000","last_active":"2023-10-06T11:22:01.612709+0000","last_peered":"2023-10-06T11:22:01.612709+0000","last_clean":"2023-10-06T11:22:01.612709+0000","last_became_active":"2023-10-06T11:21:57.576632+0000","last_became_peered":"2023-10-06T11:21:57.576632+0000","last_unstale":"2023-10-06T11:22:01.612709+0000","last_undegraded":"2023-10-06T11:22:01.612709+0000","last_fullsized":"2023-10-06T11:22:01.612709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:32.416966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617372+0000","last_change":"2023-10-06T11:21:56.575818+0000","last_active":"2023-10-06T11:22:01.617372+0000","last_peered":"2023-10-06T11:22:01.617372+0000","last_clean":"2023-10-06T11:22:01.617372+0000","last_became_active":"2023-10-06T11:21:56.575460+0000","last_became_peered":"2023-10-06T11:21:56.575460+0000","last_unstale":"2023-10-06T11:22:01.617372+0000","last_undegraded":"2023-10-06T11:22:01.617372+0000","last_fullsized":"2023-10-06T11:22:01.617372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.634446+0000","last_change":"2023-10-06T11:22:01.632266+0000","last_active":"2023-10-06T11:22:01.634446+0000","last_peered":"2023-10-06T11:22:01.634446+0000","last_clean":"2023-10-06T11:22:01.634446+0000","last_became_active":"2023-10-06T11:22:01.631851+0000","last_became_peered":"2023-10-06T11:22:01.631851+0000","last_unstale":"2023-10-06T11:22:01.634446+0000","last_undegraded":"2023-10-06T11:22:01.634446+0000","last_fullsized":"2023-10-06T11:22:01.634446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:11.811755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626480+0000","last_change":"2023-10-06T11:21:57.561599+0000","last_active":"2023-10-06T11:22:01.626480+0000","last_peered":"2023-10-06T11:22:01.626480+0000","last_clean":"2023-10-06T11:22:01.626480+0000","last_became_active":"2023-10-06T11:21:57.561280+0000","last_became_peered":"2023-10-06T11:21:57.561280+0000","last_unstale":"2023-10-06T11:22:01.626480+0000","last_undegraded":"2023-10-06T11:22:01.626480+0000","last_fullsized":"2023-10-06T11:22:01.626480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621119+0000","last_change":"2023-10-06T11:21:56.566817+0000","last_active":"2023-10-06T11:22:01.621119+0000","last_peered":"2023-10-06T11:22:01.621119+0000","last_clean":"2023-10-06T11:22:01.621119+0000","last_became_active":"2023-10-06T11:21:56.566466+0000","last_became_peered":"2023-10-06T11:21:56.566466+0000","last_unstale":"2023-10-06T11:22:01.621119+0000","last_undegraded":"2023-10-06T11:22:01.621119+0000","last_fullsized":"2023-10-06T11:22:01.621119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:30.945580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.632874+0000","last_change":"2023-10-06T11:22:01.628287+0000","last_active":"2023-10-06T11:22:01.632874+0000","last_peered":"2023-10-06T11:22:01.632874+0000","last_clean":"2023-10-06T11:22:01.632874+0000","last_became_active":"2023-10-06T11:22:01.627842+0000","last_became_peered":"2023-10-06T11:22:01.627842+0000","last_unstale":"2023-10-06T11:22:01.632874+0000","last_undegraded":"2023-10-06T11:22:01.632874+0000","last_fullsized":"2023-10-06T11:22:01.632874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:42.027969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620876+0000","last_change":"2023-10-06T11:21:57.572846+0000","last_active":"2023-10-06T11:22:01.620876+0000","last_peered":"2023-10-06T11:22:01.620876+0000","last_clean":"2023-10-06T11:22:01.620876+0000","last_became_active":"2023-10-06T11:21:57.572567+0000","last_became_peered":"2023-10-06T11:21:57.572567+0000","last_unstale":"2023-10-06T11:22:01.620876+0000","last_undegraded":"2023-10-06T11:22:01.620876+0000","last_fullsized":"2023-10-06T11:22:01.620876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620369+0000","last_change":"2023-10-06T11:21:56.575220+0000","last_active":"2023-10-06T11:22:01.620369+0000","last_peered":"2023-10-06T11:22:01.620369+0000","last_clean":"2023-10-06T11:22:01.620369+0000","last_became_active":"2023-10-06T11:21:56.574557+0000","last_became_peered":"2023-10-06T11:21:56.574557+0000","last_unstale":"2023-10-06T11:22:01.620369+0000","last_undegraded":"2023-10-06T11:22:01.620369+0000","last_fullsized":"2023-10-06T11:22:01.620369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:44.226789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.639894+0000","last_change":"2023-10-06T11:22:01.637484+0000","last_active":"2023-10-06T11:22:01.639894+0000","last_peered":"2023-10-06T11:22:01.639894+0000","last_clean":"2023-10-06T11:22:01.639894+0000","last_became_active":"2023-10-06T11:22:01.637045+0000","last_became_peered":"2023-10-06T11:22:01.637045+0000","last_unstale":"2023-10-06T11:22:01.639894+0000","last_undegraded":"2023-10-06T11:22:01.639894+0000","last_fullsized":"2023-10-06T11:22:01.639894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:36.472691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610908+0000","last_change":"2023-10-06T11:21:57.562666+0000","last_active":"2023-10-06T11:22:01.610908+0000","last_peered":"2023-10-06T11:22:01.610908+0000","last_clean":"2023-10-06T11:22:01.610908+0000","last_became_active":"2023-10-06T11:21:57.562384+0000","last_became_peered":"2023-10-06T11:21:57.562384+0000","last_unstale":"2023-10-06T11:22:01.610908+0000","last_undegraded":"2023-10-06T11:22:01.610908+0000","last_fullsized":"2023-10-06T11:22:01.610908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:03.832047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619654+0000","last_change":"2023-10-06T11:21:56.578885+0000","last_active":"2023-10-06T11:22:01.619654+0000","last_peered":"2023-10-06T11:22:01.619654+0000","last_clean":"2023-10-06T11:22:01.619654+0000","last_became_active":"2023-10-06T11:21:56.578589+0000","last_became_peered":"2023-10-06T11:21:56.578589+0000","last_unstale":"2023-10-06T11:22:01.619654+0000","last_undegraded":"2023-10-06T11:22:01.619654+0000","last_fullsized":"2023-10-06T11:22:01.619654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:31.263377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.629203+0000","last_change":"2023-10-06T11:22:01.626674+0000","last_active":"2023-10-06T11:22:01.629203+0000","last_peered":"2023-10-06T11:22:01.629203+0000","last_clean":"2023-10-06T11:22:01.629203+0000","last_became_active":"2023-10-06T11:22:01.626199+0000","last_became_peered":"2023-10-06T11:22:01.626199+0000","last_unstale":"2023-10-06T11:22:01.629203+0000","last_undegraded":"2023-10-06T11:22:01.629203+0000","last_fullsized":"2023-10-06T11:22:01.629203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:07.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619746+0000","last_change":"2023-10-06T11:21:57.568271+0000","last_active":"2023-10-06T11:22:01.619746+0000","last_peered":"2023-10-06T11:22:01.619746+0000","last_clean":"2023-10-06T11:22:01.619746+0000","last_became_active":"2023-10-06T11:21:57.567943+0000","last_became_peered":"2023-10-06T11:21:57.567943+0000","last_unstale":"2023-10-06T11:22:01.619746+0000","last_undegraded":"2023-10-06T11:22:01.619746+0000","last_fullsized":"2023-10-06T11:22:01.619746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:31.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619520+0000","last_change":"2023-10-06T11:21:56.579019+0000","last_active":"2023-10-06T11:22:01.619520+0000","last_peered":"2023-10-06T11:22:01.619520+0000","last_clean":"2023-10-06T11:22:01.619520+0000","last_became_active":"2023-10-06T11:21:56.578659+0000","last_became_peered":"2023-10-06T11:21:56.578659+0000","last_unstale":"2023-10-06T11:22:01.619520+0000","last_undegraded":"2023-10-06T11:22:01.619520+0000","last_fullsized":"2023-10-06T11:22:01.619520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:32.338385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.643049+0000","last_change":"2023-10-06T11:22:01.637818+0000","last_active":"2023-10-06T11:22:01.643049+0000","last_peered":"2023-10-06T11:22:01.643049+0000","last_clean":"2023-10-06T11:22:01.643049+0000","last_became_active":"2023-10-06T11:22:01.637325+0000","last_became_peered":"2023-10-06T11:22:01.637325+0000","last_unstale":"2023-10-06T11:22:01.643049+0000","last_undegraded":"2023-10-06T11:22:01.643049+0000","last_fullsized":"2023-10-06T11:22:01.643049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:00.090045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.641080+0000","last_change":"2023-10-06T11:22:01.636779+0000","last_active":"2023-10-06T11:22:01.641080+0000","last_peered":"2023-10-06T11:22:01.641080+0000","last_clean":"2023-10-06T11:22:01.641080+0000","last_became_active":"2023-10-06T11:22:01.636344+0000","last_became_peered":"2023-10-06T11:22:01.636344+0000","last_unstale":"2023-10-06T11:22:01.641080+0000","last_undegraded":"2023-10-06T11:22:01.641080+0000","last_fullsized":"2023-10-06T11:22:01.641080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:54.380628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620730+0000","last_change":"2023-10-06T11:21:56.575584+0000","last_active":"2023-10-06T11:22:01.620730+0000","last_peered":"2023-10-06T11:22:01.620730+0000","last_clean":"2023-10-06T11:22:01.620730+0000","last_became_active":"2023-10-06T11:21:56.575201+0000","last_became_peered":"2023-10-06T11:21:56.575201+0000","last_unstale":"2023-10-06T11:22:01.620730+0000","last_undegraded":"2023-10-06T11:22:01.620730+0000","last_fullsized":"2023-10-06T11:22:01.620730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:33.828668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616376+0000","last_change":"2023-10-06T11:21:57.568259+0000","last_active":"2023-10-06T11:22:01.616376+0000","last_peered":"2023-10-06T11:22:01.616376+0000","last_clean":"2023-10-06T11:22:01.616376+0000","last_became_active":"2023-10-06T11:21:57.567958+0000","last_became_peered":"2023-10-06T11:21:57.567958+0000","last_unstale":"2023-10-06T11:22:01.616376+0000","last_undegraded":"2023-10-06T11:22:01.616376+0000","last_fullsized":"2023-10-06T11:22:01.616376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.637658+0000","last_change":"2023-10-06T11:22:01.635766+0000","last_active":"2023-10-06T11:22:01.637658+0000","last_peered":"2023-10-06T11:22:01.637658+0000","last_clean":"2023-10-06T11:22:01.637658+0000","last_became_active":"2023-10-06T11:22:01.635314+0000","last_became_peered":"2023-10-06T11:22:01.635314+0000","last_unstale":"2023-10-06T11:22:01.637658+0000","last_undegraded":"2023-10-06T11:22:01.637658+0000","last_fullsized":"2023-10-06T11:22:01.637658+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:33:19.459200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619865+0000","last_change":"2023-10-06T11:21:56.579444+0000","last_active":"2023-10-06T11:22:01.619865+0000","last_peered":"2023-10-06T11:22:01.619865+0000","last_clean":"2023-10-06T11:22:01.619865+0000","last_became_active":"2023-10-06T11:21:56.578427+0000","last_became_peered":"2023-10-06T11:21:56.578427+0000","last_unstale":"2023-10-06T11:22:01.619865+0000","last_undegraded":"2023-10-06T11:22:01.619865+0000","last_fullsized":"2023-10-06T11:22:01.619865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:21.509181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627696+0000","last_change":"2023-10-06T11:21:57.569150+0000","last_active":"2023-10-06T11:22:01.627696+0000","last_peered":"2023-10-06T11:22:01.627696+0000","last_clean":"2023-10-06T11:22:01.627696+0000","last_became_active":"2023-10-06T11:21:57.568883+0000","last_became_peered":"2023-10-06T11:21:57.568883+0000","last_unstale":"2023-10-06T11:22:01.627696+0000","last_undegraded":"2023-10-06T11:22:01.627696+0000","last_fullsized":"2023-10-06T11:22:01.627696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:34.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626255+0000","last_change":"2023-10-06T11:21:56.572563+0000","last_active":"2023-10-06T11:22:01.626255+0000","last_peered":"2023-10-06T11:22:01.626255+0000","last_clean":"2023-10-06T11:22:01.626255+0000","last_became_active":"2023-10-06T11:21:56.571525+0000","last_became_peered":"2023-10-06T11:21:56.571525+0000","last_unstale":"2023-10-06T11:22:01.626255+0000","last_undegraded":"2023-10-06T11:22:01.626255+0000","last_fullsized":"2023-10-06T11:22:01.626255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:03.720499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621535+0000","last_change":"2023-10-06T11:21:57.569979+0000","last_active":"2023-10-06T11:22:01.621535+0000","last_peered":"2023-10-06T11:22:01.621535+0000","last_clean":"2023-10-06T11:22:01.621535+0000","last_became_active":"2023-10-06T11:21:57.569669+0000","last_became_peered":"2023-10-06T11:21:57.569669+0000","last_unstale":"2023-10-06T11:22:01.621535+0000","last_undegraded":"2023-10-06T11:22:01.621535+0000","last_fullsized":"2023-10-06T11:22:01.621535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.638165+0000","last_change":"2023-10-06T11:22:01.636951+0000","last_active":"2023-10-06T11:22:01.638165+0000","last_peered":"2023-10-06T11:22:01.638165+0000","last_clean":"2023-10-06T11:22:01.638165+0000","last_became_active":"2023-10-06T11:22:01.636534+0000","last_became_peered":"2023-10-06T11:22:01.636534+0000","last_unstale":"2023-10-06T11:22:01.638165+0000","last_undegraded":"2023-10-06T11:22:01.638165+0000","last_fullsized":"2023-10-06T11:22:01.638165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:20.016964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621855+0000","last_change":"2023-10-06T11:21:56.579219+0000","last_active":"2023-10-06T11:22:01.621855+0000","last_peered":"2023-10-06T11:22:01.621855+0000","last_clean":"2023-10-06T11:22:01.621855+0000","last_became_active":"2023-10-06T11:21:56.578100+0000","last_became_peered":"2023-10-06T11:21:56.578100+0000","last_unstale":"2023-10-06T11:22:01.621855+0000","last_undegraded":"2023-10-06T11:22:01.621855+0000","last_fullsized":"2023-10-06T11:22:01.621855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:55.528156+0000","last_clean_scrub_stamp":"2023-10-06T11:21:55.528156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:34:25.502372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616499+0000","last_change":"2023-10-06T11:21:57.580696+0000","last_active":"2023-10-06T11:22:01.616499+0000","last_peered":"2023-10-06T11:22:01.616499+0000","last_clean":"2023-10-06T11:22:01.616499+0000","last_became_active":"2023-10-06T11:21:57.580262+0000","last_became_peered":"2023-10-06T11:21:57.580262+0000","last_unstale":"2023-10-06T11:22:01.616499+0000","last_undegraded":"2023-10-06T11:22:01.616499+0000","last_fullsized":"2023-10-06T11:22:01.616499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.557769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.635928+0000","last_change":"2023-10-06T11:22:01.632977+0000","last_active":"2023-10-06T11:22:01.635928+0000","last_peered":"2023-10-06T11:22:01.635928+0000","last_clean":"2023-10-06T11:22:01.635928+0000","last_became_active":"2023-10-06T11:22:01.632522+0000","last_became_peered":"2023-10-06T11:22:01.632522+0000","last_unstale":"2023-10-06T11:22:01.635928+0000","last_undegraded":"2023-10-06T11:22:01.635928+0000","last_fullsized":"2023-10-06T11:22:01.635928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:22:00.588605+0000","last_clean_scrub_stamp":"2023-10-06T11:22:00.588605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:30:24.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.620591+0000","last_change":"2023-10-06T11:21:57.570931+0000","last_active":"2023-10-06T11:22:01.620591+0000","last_peered":"2023-10-06T11:22:01.620591+0000","last_clean":"2023-10-06T11:22:01.620591+0000","last_became_active":"2023-10-06T11:21:57.570619+0000","last_became_peered":"2023-10-06T11:21:57.570619+0000","last_unstale":"2023-10-06T11:22:01.620591+0000","last_undegraded":"2023-10-06T11:22:01.620591+0000","last_fullsized":"2023-10-06T11:22:01.620591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:03.048434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615615+0000","last_change":"2023-10-06T11:21:57.572767+0000","last_active":"2023-10-06T11:22:01.615615+0000","last_peered":"2023-10-06T11:22:01.615615+0000","last_clean":"2023-10-06T11:22:01.615615+0000","last_became_active":"2023-10-06T11:21:57.572371+0000","last_became_peered":"2023-10-06T11:21:57.572371+0000","last_unstale":"2023-10-06T11:22:01.615615+0000","last_undegraded":"2023-10-06T11:22:01.615615+0000","last_fullsized":"2023-10-06T11:22:01.615615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610290+0000","last_change":"2023-10-06T11:21:57.571303+0000","last_active":"2023-10-06T11:22:01.610290+0000","last_peered":"2023-10-06T11:22:01.610290+0000","last_clean":"2023-10-06T11:22:01.610290+0000","last_became_active":"2023-10-06T11:21:57.570978+0000","last_became_peered":"2023-10-06T11:21:57.570978+0000","last_unstale":"2023-10-06T11:22:01.610290+0000","last_undegraded":"2023-10-06T11:22:01.610290+0000","last_fullsized":"2023-10-06T11:22:01.610290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:42.556757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.617222+0000","last_change":"2023-10-06T11:21:57.569031+0000","last_active":"2023-10-06T11:22:01.617222+0000","last_peered":"2023-10-06T11:22:01.617222+0000","last_clean":"2023-10-06T11:22:01.617222+0000","last_became_active":"2023-10-06T11:21:57.568734+0000","last_became_peered":"2023-10-06T11:21:57.568734+0000","last_unstale":"2023-10-06T11:22:01.617222+0000","last_undegraded":"2023-10-06T11:22:01.617222+0000","last_fullsized":"2023-10-06T11:22:01.617222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619002+0000","last_change":"2023-10-06T11:21:57.579632+0000","last_active":"2023-10-06T11:22:01.619002+0000","last_peered":"2023-10-06T11:22:01.619002+0000","last_clean":"2023-10-06T11:22:01.619002+0000","last_became_active":"2023-10-06T11:21:57.567143+0000","last_became_peered":"2023-10-06T11:21:57.567143+0000","last_unstale":"2023-10-06T11:22:01.619002+0000","last_undegraded":"2023-10-06T11:22:01.619002+0000","last_fullsized":"2023-10-06T11:22:01.619002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621442+0000","last_change":"2023-10-06T11:21:57.569847+0000","last_active":"2023-10-06T11:22:01.621442+0000","last_peered":"2023-10-06T11:22:01.621442+0000","last_clean":"2023-10-06T11:22:01.621442+0000","last_became_active":"2023-10-06T11:21:57.569571+0000","last_became_peered":"2023-10-06T11:21:57.569571+0000","last_unstale":"2023-10-06T11:22:01.621442+0000","last_undegraded":"2023-10-06T11:22:01.621442+0000","last_fullsized":"2023-10-06T11:22:01.621442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621862+0000","last_change":"2023-10-06T11:21:57.562969+0000","last_active":"2023-10-06T11:22:01.621862+0000","last_peered":"2023-10-06T11:22:01.621862+0000","last_clean":"2023-10-06T11:22:01.621862+0000","last_became_active":"2023-10-06T11:21:57.562623+0000","last_became_peered":"2023-10-06T11:21:57.562623+0000","last_unstale":"2023-10-06T11:22:01.621862+0000","last_undegraded":"2023-10-06T11:22:01.621862+0000","last_fullsized":"2023-10-06T11:22:01.621862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.614335+0000","last_change":"2023-10-06T11:21:57.571639+0000","last_active":"2023-10-06T11:22:01.614335+0000","last_peered":"2023-10-06T11:22:01.614335+0000","last_clean":"2023-10-06T11:22:01.614335+0000","last_became_active":"2023-10-06T11:21:57.571347+0000","last_became_peered":"2023-10-06T11:21:57.571347+0000","last_unstale":"2023-10-06T11:22:01.614335+0000","last_undegraded":"2023-10-06T11:22:01.614335+0000","last_fullsized":"2023-10-06T11:22:01.614335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:28:40.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.616721+0000","last_change":"2023-10-06T11:21:57.576904+0000","last_active":"2023-10-06T11:22:01.616721+0000","last_peered":"2023-10-06T11:22:01.616721+0000","last_clean":"2023-10-06T11:22:01.616721+0000","last_became_active":"2023-10-06T11:21:57.576570+0000","last_became_peered":"2023-10-06T11:21:57.576570+0000","last_unstale":"2023-10-06T11:22:01.616721+0000","last_undegraded":"2023-10-06T11:22:01.616721+0000","last_fullsized":"2023-10-06T11:22:01.616721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.615508+0000","last_change":"2023-10-06T11:21:57.572666+0000","last_active":"2023-10-06T11:22:01.615508+0000","last_peered":"2023-10-06T11:22:01.615508+0000","last_clean":"2023-10-06T11:22:01.615508+0000","last_became_active":"2023-10-06T11:21:57.569109+0000","last_became_peered":"2023-10-06T11:21:57.569109+0000","last_unstale":"2023-10-06T11:22:01.615508+0000","last_undegraded":"2023-10-06T11:22:01.615508+0000","last_fullsized":"2023-10-06T11:22:01.615508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626091+0000","last_change":"2023-10-06T11:21:57.566406+0000","last_active":"2023-10-06T11:22:01.626091+0000","last_peered":"2023-10-06T11:22:01.626091+0000","last_clean":"2023-10-06T11:22:01.626091+0000","last_became_active":"2023-10-06T11:21:57.565954+0000","last_became_peered":"2023-10-06T11:21:57.565954+0000","last_unstale":"2023-10-06T11:22:01.626091+0000","last_undegraded":"2023-10-06T11:22:01.626091+0000","last_fullsized":"2023-10-06T11:22:01.626091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:56.155493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621753+0000","last_change":"2023-10-06T11:21:57.576914+0000","last_active":"2023-10-06T11:22:01.621753+0000","last_peered":"2023-10-06T11:22:01.621753+0000","last_clean":"2023-10-06T11:22:01.621753+0000","last_became_active":"2023-10-06T11:21:57.576444+0000","last_became_peered":"2023-10-06T11:21:57.576444+0000","last_unstale":"2023-10-06T11:22:01.621753+0000","last_undegraded":"2023-10-06T11:22:01.621753+0000","last_fullsized":"2023-10-06T11:22:01.621753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612641+0000","last_change":"2023-10-06T11:21:57.565842+0000","last_active":"2023-10-06T11:22:01.612641+0000","last_peered":"2023-10-06T11:22:01.612641+0000","last_clean":"2023-10-06T11:22:01.612641+0000","last_became_active":"2023-10-06T11:21:57.565455+0000","last_became_peered":"2023-10-06T11:21:57.565455+0000","last_unstale":"2023-10-06T11:22:01.612641+0000","last_undegraded":"2023-10-06T11:22:01.612641+0000","last_fullsized":"2023-10-06T11:22:01.612641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:28.522557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612331+0000","last_change":"2023-10-06T11:21:57.569901+0000","last_active":"2023-10-06T11:22:01.612331+0000","last_peered":"2023-10-06T11:22:01.612331+0000","last_clean":"2023-10-06T11:22:01.612331+0000","last_became_active":"2023-10-06T11:21:57.569642+0000","last_became_peered":"2023-10-06T11:21:57.569642+0000","last_unstale":"2023-10-06T11:22:01.612331+0000","last_undegraded":"2023-10-06T11:22:01.612331+0000","last_fullsized":"2023-10-06T11:22:01.612331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:31.950303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.611118+0000","last_change":"2023-10-06T11:21:57.564154+0000","last_active":"2023-10-06T11:22:01.611118+0000","last_peered":"2023-10-06T11:22:01.611118+0000","last_clean":"2023-10-06T11:22:01.611118+0000","last_became_active":"2023-10-06T11:21:57.563840+0000","last_became_peered":"2023-10-06T11:21:57.563840+0000","last_unstale":"2023-10-06T11:22:01.611118+0000","last_undegraded":"2023-10-06T11:22:01.611118+0000","last_fullsized":"2023-10-06T11:22:01.611118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627637+0000","last_change":"2023-10-06T11:21:57.569223+0000","last_active":"2023-10-06T11:22:01.627637+0000","last_peered":"2023-10-06T11:22:01.627637+0000","last_clean":"2023-10-06T11:22:01.627637+0000","last_became_active":"2023-10-06T11:21:57.568967+0000","last_became_peered":"2023-10-06T11:21:57.568967+0000","last_unstale":"2023-10-06T11:22:01.627637+0000","last_undegraded":"2023-10-06T11:22:01.627637+0000","last_fullsized":"2023-10-06T11:22:01.627637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.612585+0000","last_change":"2023-10-06T11:21:57.567578+0000","last_active":"2023-10-06T11:22:01.612585+0000","last_peered":"2023-10-06T11:22:01.612585+0000","last_clean":"2023-10-06T11:22:01.612585+0000","last_became_active":"2023-10-06T11:21:57.565977+0000","last_became_peered":"2023-10-06T11:21:57.565977+0000","last_unstale":"2023-10-06T11:22:01.612585+0000","last_undegraded":"2023-10-06T11:22:01.612585+0000","last_fullsized":"2023-10-06T11:22:01.612585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:07.756030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619766+0000","last_change":"2023-10-06T11:21:57.576090+0000","last_active":"2023-10-06T11:22:01.619766+0000","last_peered":"2023-10-06T11:22:01.619766+0000","last_clean":"2023-10-06T11:22:01.619766+0000","last_became_active":"2023-10-06T11:21:57.575667+0000","last_became_peered":"2023-10-06T11:21:57.575667+0000","last_unstale":"2023-10-06T11:22:01.619766+0000","last_undegraded":"2023-10-06T11:22:01.619766+0000","last_fullsized":"2023-10-06T11:22:01.619766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:36.692922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621178+0000","last_change":"2023-10-06T11:21:57.563777+0000","last_active":"2023-10-06T11:22:01.621178+0000","last_peered":"2023-10-06T11:22:01.621178+0000","last_clean":"2023-10-06T11:22:01.621178+0000","last_became_active":"2023-10-06T11:21:57.563485+0000","last_became_peered":"2023-10-06T11:21:57.563485+0000","last_unstale":"2023-10-06T11:22:01.621178+0000","last_undegraded":"2023-10-06T11:22:01.621178+0000","last_fullsized":"2023-10-06T11:22:01.621178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:01:31.457077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610375+0000","last_change":"2023-10-06T11:21:57.575519+0000","last_active":"2023-10-06T11:22:01.610375+0000","last_peered":"2023-10-06T11:22:01.610375+0000","last_clean":"2023-10-06T11:22:01.610375+0000","last_became_active":"2023-10-06T11:21:57.575122+0000","last_became_peered":"2023-10-06T11:21:57.575122+0000","last_unstale":"2023-10-06T11:22:01.610375+0000","last_undegraded":"2023-10-06T11:22:01.610375+0000","last_fullsized":"2023-10-06T11:22:01.610375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:01.582258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607465+0000","last_change":"2023-10-06T11:21:57.563806+0000","last_active":"2023-10-06T11:22:01.607465+0000","last_peered":"2023-10-06T11:22:01.607465+0000","last_clean":"2023-10-06T11:22:01.607465+0000","last_became_active":"2023-10-06T11:21:57.561796+0000","last_became_peered":"2023-10-06T11:21:57.561796+0000","last_unstale":"2023-10-06T11:22:01.607465+0000","last_undegraded":"2023-10-06T11:22:01.607465+0000","last_fullsized":"2023-10-06T11:22:01.607465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.610200+0000","last_change":"2023-10-06T11:21:57.560661+0000","last_active":"2023-10-06T11:22:01.610200+0000","last_peered":"2023-10-06T11:22:01.610200+0000","last_clean":"2023-10-06T11:22:01.610200+0000","last_became_active":"2023-10-06T11:21:57.559848+0000","last_became_peered":"2023-10-06T11:21:57.559848+0000","last_unstale":"2023-10-06T11:22:01.610200+0000","last_undegraded":"2023-10-06T11:22:01.610200+0000","last_fullsized":"2023-10-06T11:22:01.610200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:37.588925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621548+0000","last_change":"2023-10-06T11:21:57.566423+0000","last_active":"2023-10-06T11:22:01.621548+0000","last_peered":"2023-10-06T11:22:01.621548+0000","last_clean":"2023-10-06T11:22:01.621548+0000","last_became_active":"2023-10-06T11:21:57.566064+0000","last_became_peered":"2023-10-06T11:21:57.566064+0000","last_unstale":"2023-10-06T11:22:01.621548+0000","last_undegraded":"2023-10-06T11:22:01.621548+0000","last_fullsized":"2023-10-06T11:22:01.621548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:37.972244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.627537+0000","last_change":"2023-10-06T11:21:57.575695+0000","last_active":"2023-10-06T11:22:01.627537+0000","last_peered":"2023-10-06T11:22:01.627537+0000","last_clean":"2023-10-06T11:22:01.627537+0000","last_became_active":"2023-10-06T11:21:57.575418+0000","last_became_peered":"2023-10-06T11:21:57.575418+0000","last_unstale":"2023-10-06T11:22:01.627537+0000","last_undegraded":"2023-10-06T11:22:01.627537+0000","last_fullsized":"2023-10-06T11:22:01.627537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:13.245520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619861+0000","last_change":"2023-10-06T11:21:57.562165+0000","last_active":"2023-10-06T11:22:01.619861+0000","last_peered":"2023-10-06T11:22:01.619861+0000","last_clean":"2023-10-06T11:22:01.619861+0000","last_became_active":"2023-10-06T11:21:57.561878+0000","last_became_peered":"2023-10-06T11:21:57.561878+0000","last_unstale":"2023-10-06T11:22:01.619861+0000","last_undegraded":"2023-10-06T11:22:01.619861+0000","last_fullsized":"2023-10-06T11:22:01.619861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:10:07.738003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619664+0000","last_change":"2023-10-06T11:21:57.575950+0000","last_active":"2023-10-06T11:22:01.619664+0000","last_peered":"2023-10-06T11:22:01.619664+0000","last_clean":"2023-10-06T11:22:01.619664+0000","last_became_active":"2023-10-06T11:21:57.575280+0000","last_became_peered":"2023-10-06T11:21:57.575280+0000","last_unstale":"2023-10-06T11:22:01.619664+0000","last_undegraded":"2023-10-06T11:22:01.619664+0000","last_fullsized":"2023-10-06T11:22:01.619664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609933+0000","last_change":"2023-10-06T11:21:57.566684+0000","last_active":"2023-10-06T11:22:01.609933+0000","last_peered":"2023-10-06T11:22:01.609933+0000","last_clean":"2023-10-06T11:22:01.609933+0000","last_became_active":"2023-10-06T11:21:57.566358+0000","last_became_peered":"2023-10-06T11:21:57.566358+0000","last_unstale":"2023-10-06T11:22:01.609933+0000","last_undegraded":"2023-10-06T11:22:01.609933+0000","last_fullsized":"2023-10-06T11:22:01.609933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:50.324902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.626719+0000","last_change":"2023-10-06T11:21:57.574099+0000","last_active":"2023-10-06T11:22:01.626719+0000","last_peered":"2023-10-06T11:22:01.626719+0000","last_clean":"2023-10-06T11:22:01.626719+0000","last_became_active":"2023-10-06T11:21:57.573592+0000","last_became_peered":"2023-10-06T11:21:57.573592+0000","last_unstale":"2023-10-06T11:22:01.626719+0000","last_undegraded":"2023-10-06T11:22:01.626719+0000","last_fullsized":"2023-10-06T11:22:01.626719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:06.566500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.619959+0000","last_change":"2023-10-06T11:21:57.566310+0000","last_active":"2023-10-06T11:22:01.619959+0000","last_peered":"2023-10-06T11:22:01.619959+0000","last_clean":"2023-10-06T11:22:01.619959+0000","last_became_active":"2023-10-06T11:21:57.566014+0000","last_became_peered":"2023-10-06T11:21:57.566014+0000","last_unstale":"2023-10-06T11:22:01.619959+0000","last_undegraded":"2023-10-06T11:22:01.619959+0000","last_fullsized":"2023-10-06T11:22:01.619959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:32.268101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.609857+0000","last_change":"2023-10-06T11:21:57.567982+0000","last_active":"2023-10-06T11:22:01.609857+0000","last_peered":"2023-10-06T11:22:01.609857+0000","last_clean":"2023-10-06T11:22:01.609857+0000","last_became_active":"2023-10-06T11:21:57.567667+0000","last_became_peered":"2023-10-06T11:21:57.567667+0000","last_unstale":"2023-10-06T11:22:01.609857+0000","last_undegraded":"2023-10-06T11:22:01.609857+0000","last_fullsized":"2023-10-06T11:22:01.609857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:14.276554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.621434+0000","last_change":"2023-10-06T11:21:57.562657+0000","last_active":"2023-10-06T11:22:01.621434+0000","last_peered":"2023-10-06T11:22:01.621434+0000","last_clean":"2023-10-06T11:22:01.621434+0000","last_became_active":"2023-10-06T11:21:57.562338+0000","last_became_peered":"2023-10-06T11:21:57.562338+0000","last_unstale":"2023-10-06T11:22:01.621434+0000","last_undegraded":"2023-10-06T11:22:01.621434+0000","last_fullsized":"2023-10-06T11:22:01.621434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:05.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T11:22:01.607967+0000","last_change":"2023-10-06T11:21:57.569451+0000","last_active":"2023-10-06T11:22:01.607967+0000","last_peered":"2023-10-06T11:22:01.607967+0000","last_clean":"2023-10-06T11:22:01.607967+0000","last_became_active":"2023-10-06T11:21:57.569124+0000","last_became_peered":"2023-10-06T11:21:57.569124+0000","last_unstale":"2023-10-06T11:22:01.607967+0000","last_undegraded":"2023-10-06T11:22:01.607967+0000","last_fullsized":"2023-10-06T11:22:01.607967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:56.532879+0000","last_clean_scrub_stamp":"2023-10-06T11:21:56.532879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.833392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":703756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301652,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070188,"statfs":{"total":96636764160,"available":96327872512,"internally_reserved":0,"allocated":274432,"data_stored":110082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":286720,"data_stored":113228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":109548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301644,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070196,"statfs":{"total":96636764160,"available":96327880704,"internally_reserved":0,"allocated":266240,"data_stored":109458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302224,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069616,"statfs":{"total":96636764160,"available":96327286784,"internally_reserved":0,"allocated":860160,"data_stored":699826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":270336,"data_stored":110012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:22:21.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T11:22:21.593 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T11:22:21.593 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T11:22:21.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T11:22:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 Processor -- start 2023-10-06T11:22:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 -- start start 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f185c0cab80 0x7f185c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0ca0a0 con 0x7f185c0ad740 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c157230 con 0x7f185c0afb00 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.683+0000 7f1863231700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c0ae2d0 con 0x7f185c0cab80 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1862a30700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f185c0ad740 0x7f185c0adbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54966/0 (socket says 172.21.15.7:54966) 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1862a30700 1 -- 172.21.15.7:0/1124721810 learned_addr learned my addr 172.21.15.7:0/1124721810 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1861a2e700 1 --2- 172.21.15.7:0/1124721810 >> v2:172.21.15.50:3300/0 conn(0x7f185c0cab80 0x7f185c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365370965 0 0) 0x7f185c0ca0a0 con 0x7f185c0ad740 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1848003570 con 0x7f185c0ad740 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1861a2e700 1 -- 172.21.15.7:0/1124721810 >> v1:172.21.15.50:6789/0 conn(0x7f185c0afb00 legacy=0x7f185c0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1861a2e700 1 -- 172.21.15.7:0/1124721810 >> v1:172.21.15.7:6789/0 conn(0x7f185c0ad740 legacy=0x7f185c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2024007327 0 0) 0x7f1848003570 con 0x7f185c0ad740 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1861a2e700 1 -- 172.21.15.7:0/1124721810 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c157510 con 0x7f185c0cab80 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1861a2e700 1 --2- 172.21.15.7:0/1124721810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0cab80 0x7f185c0afed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098e0 tx=0x7f184c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d6c4cf3ecd29c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540017f0 con 0x7f185c0cab80 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c011420 con 0x7f185c0cab80 2023-10-06T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f186122d700 1 -- 172.21.15.7:0/1124721810 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c0117c0 con 0x7f185c0cab80 2023-10-06T11:22:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- 172.21.15.7:0/1124721810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0cab80 msgr2=0x7f185c0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 --2- 172.21.15.7:0/1124721810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0cab80 0x7f185c0afed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098e0 tx=0x7f184c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- 172.21.15.7:0/1124721810 shutdown_connections 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 --2- 172.21.15.7:0/1124721810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0cab80 0x7f185c0afed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- 172.21.15.7:0/1124721810 >> 172.21.15.7:0/1124721810 conn(0x7f185c06bd70 msgr2=0x7f185c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- 172.21.15.7:0/1124721810 shutdown_connections 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- 172.21.15.7:0/1124721810 wait complete. 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 Processor -- start 2023-10-06T11:22:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.687+0000 7f1863231700 1 -- start start 2023-10-06T11:22:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0ad740 0x7f185c167370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f185c0cab80 0x7f185c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0c2f40 con 0x7f185c0afb00 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0c6830 con 0x7f185c0ad740 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0c25a0 con 0x7f185c0cab80 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41906/0 (socket says 172.21.15.7:41906) 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 -- 172.21.15.7:0/147699044 learned_addr learned my addr 172.21.15.7:0/147699044 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1862a30700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f185c0cab80 0x7f185c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f186222f700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0ad740 0x7f185c167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 -- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f185c0cab80 msgr2=0x7f185c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f185c0cab80 0x7f185c0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 -- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0ad740 msgr2=0x7f185c167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0ad740 0x7f185c167370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 -- 172.21.15.7:0/147699044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009420 con 0x7f185c0afb00 2023-10-06T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1861a2e700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f184c0091e0 tx=0x7f184c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c01cc00 con 0x7f185c0afb00 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c01cd80 con 0x7f185c0afb00 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c01a6a0 con 0x7f185c0afb00 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c0c2880 con 0x7f185c0afb00 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c1678b0 con 0x7f185c0afb00 2023-10-06T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.691+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c06ec60 con 0x7f185c0afb00 2023-10-06T11:22:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.695+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f184c01aa10 con 0x7f185c0afb00 2023-10-06T11:22:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.695+0000 7f18527fc700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 0x7f1840062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.695+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f184c07e870 con 0x7f185c0afb00 2023-10-06T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.695+0000 7f186222f700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 0x7f1840062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.695+0000 7f186222f700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 0x7f1840062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1858003a80 tx=0x7f1858007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.699+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f184c04ea10 con 0x7f185c0afb00 2023-10-06T11:22:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f185c0c2d00 con 0x7f185c0afb00 2023-10-06T11:22:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f18527fc700 1 -- 172.21.15.7:0/147699044 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f184c052030 con 0x7f185c0afb00 2023-10-06T11:22:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:21.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 msgr2=0x7f1840062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 0x7f1840062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1858003a80 tx=0x7f1858007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 msgr2=0x7f185c169fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f184c0091e0 tx=0x7f184c016000 comp rx=0 tx=0).stop 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 shutdown_connections 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1840060550 0x7f1840062a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f185c0cab80 0x7f185c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f185c0afb00 0x7f185c169fa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 --2- 172.21.15.7:0/147699044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f185c0ad740 0x7f185c167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 >> 172.21.15.7:0/147699044 conn(0x7f185c06bd70 msgr2=0x7f185c0cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 shutdown_connections 2023-10-06T11:22:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.891+0000 7f1863231700 1 -- 172.21.15.7:0/147699044 wait complete. 2023-10-06T11:22:21.909 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T11:22:21.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.999+0000 7f76409ac700 1 Processor -- start 2023-10-06T11:22:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:21.999+0000 7f76409ac700 1 -- start start 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 0x7f763c15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0aa220 con 0x7f763c15cb80 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c15af80 con 0x7f763c0a9490 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0aa020 con 0x7f763c153530 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f763c15cb80 0x7f763c152e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34988/0 (socket says 172.21.15.7:34988) 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763b7fe700 1 -- 172.21.15.7:0/3583213083 learned_addr learned my addr 172.21.15.7:0/3583213083 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763affd700 1 --2- 172.21.15.7:0/3583213083 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 0x7f763c15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763a7fc700 1 -- 172.21.15.7:0/3583213083 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803722045 0 0) 0x7f763c0aa220 con 0x7f763c15cb80 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763a7fc700 1 -- 172.21.15.7:0/3583213083 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f761c003570 con 0x7f763c15cb80 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763affd700 1 -- 172.21.15.7:0/3583213083 >> v1:172.21.15.50:6789/0 conn(0x7f763c0a9490 legacy=0x7f763c0a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763affd700 1 -- 172.21.15.7:0/3583213083 >> v1:172.21.15.7:6789/0 conn(0x7f763c15cb80 legacy=0x7f763c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763affd700 1 -- 172.21.15.7:0/3583213083 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c15a650 con 0x7f763c153530 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763affd700 1 --2- 172.21.15.7:0/3583213083 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 0x7f763c15cf50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7624009910 tx=0x7f76240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb36bbb31fe1e99d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763a7fc700 1 -- 172.21.15.7:0/3583213083 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400b680 con 0x7f763c153530 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763a7fc700 1 -- 172.21.15.7:0/3583213083 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762400b7e0 con 0x7f763c153530 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f763a7fc700 1 -- 172.21.15.7:0/3583213083 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400bb60 con 0x7f763c153530 2023-10-06T11:22:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- 172.21.15.7:0/3583213083 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 msgr2=0x7f763c15cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 --2- 172.21.15.7:0/3583213083 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 0x7f763c15cf50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7624009910 tx=0x7f76240091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- 172.21.15.7:0/3583213083 shutdown_connections 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 --2- 172.21.15.7:0/3583213083 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c153530 0x7f763c15cf50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- 172.21.15.7:0/3583213083 >> 172.21.15.7:0/3583213083 conn(0x7f763c0688a0 msgr2=0x7f763c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- 172.21.15.7:0/3583213083 shutdown_connections 2023-10-06T11:22:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.003+0000 7f76409ac700 1 -- 172.21.15.7:0/3583213083 wait complete. 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 Processor -- start 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- start start 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c0a9490 0x7f763c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 0x7f763c153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0c9e70 con 0x7f763c153530 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c154820 con 0x7f763c15cb80 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0c9260 con 0x7f763c0a9490 2023-10-06T11:22:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39918/0 (socket says 172.21.15.7:39918) 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 -- 172.21.15.7:0/3735748619 learned_addr learned my addr 172.21.15.7:0/3735748619 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763b7fe700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c0a9490 0x7f763c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763bfff700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 0x7f763c153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 -- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c0a9490 msgr2=0x7f763c0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c0a9490 0x7f763c0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 -- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 msgr2=0x7f763c153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 0x7f763c153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 -- 172.21.15.7:0/3735748619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624009420 con 0x7f763c153530 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763bfff700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 0x7f763c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f763affd700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76240091e0 tx=0x7f7624017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400bb60 con 0x7f763c153530 2023-10-06T11:22:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624000cc0 con 0x7f763c153530 2023-10-06T11:22:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c0c9540 con 0x7f763c153530 2023-10-06T11:22:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762401a680 con 0x7f763c153530 2023-10-06T11:22:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c0c9b00 con 0x7f763c153530 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c0c9fe0 con 0x7f763c153530 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.007+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f762401aab0 con 0x7f763c153530 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.011+0000 7f76337fe700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 0x7f7620062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.011+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f762407e920 con 0x7f763c153530 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.011+0000 7f763b7fe700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 0x7f7620062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.011+0000 7f763b7fe700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 0x7f7620062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f763c152f90 tx=0x7f762c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.019+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f762404eac0 con 0x7f763c153530 2023-10-06T11:22:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.171+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f763c04b5c0 con 0x7f763c153530 2023-10-06T11:22:22.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76337fe700 1 -- 172.21.15.7:0/3735748619 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f76240520e0 con 0x7f763c153530 2023-10-06T11:22:22.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:22.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:22.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 msgr2=0x7f7620062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 0x7f7620062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f763c152f90 tx=0x7f762c009040 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 msgr2=0x7f763c0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76240091e0 tx=0x7f7624017000 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 shutdown_connections 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7620060550 0x7f7620062a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f763c15cb80 0x7f763c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f763c153530 0x7f763c0cab40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 --2- 172.21.15.7:0/3735748619 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f763c0a9490 0x7f763c0c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 >> 172.21.15.7:0/3735748619 conn(0x7f763c0688a0 msgr2=0x7f763c0aa460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 shutdown_connections 2023-10-06T11:22:22.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.175+0000 7f76409ac700 1 -- 172.21.15.7:0/3735748619 wait complete. 2023-10-06T11:22:22.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:22.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:22.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 Processor -- start 2023-10-06T11:22:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 -- start start 2023-10-06T11:22:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b4006b8a0 0x7f2b4006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b4015c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b400c7680 con 0x7f2b400c7f60 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b40159680 con 0x7f2b4006b8a0 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.307+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4006c1b0 con 0x7f2b400c7b10 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b4006b8a0 0x7f2b4006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b4006b8a0 0x7f2b4006bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 -- 172.21.15.7:0/1629072222 learned_addr learned my addr 172.21.15.7:0/1629072222 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3dd9b700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b4015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 -- 172.21.15.7:0/1629072222 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 msgr2=0x7f2b4015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b4015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 -- 172.21.15.7:0/1629072222 >> v1:172.21.15.7:6789/0 conn(0x7f2b400c7f60 legacy=0x7f2b40068290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 -- 172.21.15.7:0/1629072222 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4015d4b0 con 0x7f2b4006b8a0 2023-10-06T11:22:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3dd9b700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b4015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3e59c700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4006b8a0 0x7f2b4006bc70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009f20 tx=0x7f2b38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d0ddd405e54af0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3d59a700 1 -- 172.21.15.7:0/1629072222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800e070 con 0x7f2b4006b8a0 2023-10-06T11:22:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3d59a700 1 -- 172.21.15.7:0/1629072222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800ba80 con 0x7f2b4006b8a0 2023-10-06T11:22:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3d59a700 1 -- 172.21.15.7:0/1629072222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380092e0 con 0x7f2b4006b8a0 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 -- 172.21.15.7:0/1629072222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4006b8a0 msgr2=0x7f2b4006bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4006b8a0 0x7f2b4006bc70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009f20 tx=0x7f2b38009840 comp rx=0 tx=0).stop 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 -- 172.21.15.7:0/1629072222 shutdown_connections 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b4015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/1629072222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4006b8a0 0x7f2b4006bc70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 -- 172.21.15.7:0/1629072222 >> 172.21.15.7:0/1629072222 conn(0x7f2b40066840 msgr2=0x7f2b40066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 -- 172.21.15.7:0/1629072222 shutdown_connections 2023-10-06T11:22:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.311+0000 7f2b3f59e700 1 -- 172.21.15.7:0/1629072222 wait complete. 2023-10-06T11:22:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 Processor -- start 2023-10-06T11:22:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 -- start start 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b4006b8a0 0x7f2b400ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b400cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400a9ee0 con 0x7f2b400c7f60 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c57b0 con 0x7f2b4006b8a0 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400c3160 con 0x7f2b400c7b10 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3e59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b4006b8a0 0x7f2b400ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3dd9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b400cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39922/0 (socket says 172.21.15.7:39922) 2023-10-06T11:22:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.315+0000 7f2b3ed9d700 1 -- 172.21.15.7:0/4019295157 learned_addr learned my addr 172.21.15.7:0/4019295157 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 -- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 msgr2=0x7f2b400cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b400cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 -- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b4006b8a0 msgr2=0x7f2b400ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b4006b8a0 0x7f2b400ab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 -- 172.21.15.7:0/4019295157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009ae0 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3ed9d700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c5c0 tx=0x7f2b30007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30011590 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300116f0 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b400c4e80 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30011ab0 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400c54c0 con 0x7f2b400c7f60 2023-10-06T11:22:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b400c5940 con 0x7f2b400c7f60 2023-10-06T11:22:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.319+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2b3001b3b0 con 0x7f2b400c7f60 2023-10-06T11:22:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.323+0000 7f2b2e7fc700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 0x7f2b20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.323+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2b3007bfe0 con 0x7f2b400c7f60 2023-10-06T11:22:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.323+0000 7f2b3e59c700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 0x7f2b20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.323+0000 7f2b3e59c700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 0x7f2b20062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009880 tx=0x7f2b38018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.331+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2b3004c180 con 0x7f2b400c7f60 2023-10-06T11:22:22.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.491+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b4006a200 con 0x7f2b400c7f60 2023-10-06T11:22:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.491+0000 7f2b2e7fc700 1 -- 172.21.15.7:0/4019295157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b300170a0 con 0x7f2b400c7f60 2023-10-06T11:22:22.497 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 msgr2=0x7f2b20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 0x7f2b20062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009880 tx=0x7f2b38018000 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 msgr2=0x7f2b400a9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c5c0 tx=0x7f2b30007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 shutdown_connections 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2b20060550 0x7f2b20062a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b400c7f60 0x7f2b400a9370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2b400c7b10 0x7f2b400cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 --2- 172.21.15.7:0/4019295157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b4006b8a0 0x7f2b400ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 >> 172.21.15.7:0/4019295157 conn(0x7f2b40066840 msgr2=0x7f2b4015c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 shutdown_connections 2023-10-06T11:22:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.495+0000 7f2b3f59e700 1 -- 172.21.15.7:0/4019295157 wait complete. 2023-10-06T11:22:22.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 Processor -- start 2023-10-06T11:22:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 -- start start 2023-10-06T11:22:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 0x7fa89815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 0x7fa8980aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980ac0b0 con 0x7fa8980aac00 2023-10-06T11:22:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980ab1c0 con 0x7fa8980ae990 2023-10-06T11:22:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89e0c3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980ab4a0 con 0x7fa89815cb70 2023-10-06T11:22:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d8c2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8980aac00 0x7fa8980ac540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35012/0 (socket says 172.21.15.7:35012) 2023-10-06T11:22:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d8c2700 1 -- 172.21.15.7:0/3826705311 learned_addr learned my addr 172.21.15.7:0/3826705311 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/3826705311 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 0x7fa89815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89c8c0700 1 --2- 172.21.15.7:0/3826705311 >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 0x7fa8980aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa88ffff700 1 -- 172.21.15.7:0/3826705311 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060439821 0 0) 0x7fa8980ac0b0 con 0x7fa8980aac00 2023-10-06T11:22:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa88ffff700 1 -- 172.21.15.7:0/3826705311 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa880003570 con 0x7fa8980aac00 2023-10-06T11:22:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 -- 172.21.15.7:0/3826705311 >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 msgr2=0x7fa8980aa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/3826705311 >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 0x7fa8980aa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 -- 172.21.15.7:0/3826705311 >> v1:172.21.15.7:6789/0 conn(0x7fa8980aac00 legacy=0x7fa8980ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 -- 172.21.15.7:0/3826705311 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980ab780 con 0x7fa89815cb70 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89c8c0700 1 --2- 172.21.15.7:0/3826705311 >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 0x7fa8980aa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/3826705311 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 0x7fa89815cf40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8880098e0 tx=0x7fa8880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff89843b3c6e83f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.619+0000 7fa88ffff700 1 -- 172.21.15.7:0/3826705311 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011630 con 0x7fa89815cb70 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa88ffff700 1 -- 172.21.15.7:0/3826705311 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888011790 con 0x7fa89815cb70 2023-10-06T11:22:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa88ffff700 1 -- 172.21.15.7:0/3826705311 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011b10 con 0x7fa89815cb70 2023-10-06T11:22:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 -- 172.21.15.7:0/3826705311 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 msgr2=0x7fa89815cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/3826705311 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 0x7fa89815cf40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8880098e0 tx=0x7fa8880091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 -- 172.21.15.7:0/3826705311 shutdown_connections 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/3826705311 >> v2:172.21.15.50:3300/0 conn(0x7fa8980ae990 0x7fa8980aa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/3826705311 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa89815cb70 0x7fa89815cf40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 -- 172.21.15.7:0/3826705311 >> 172.21.15.7:0/3826705311 conn(0x7fa8980cc3c0 msgr2=0x7fa898067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 -- 172.21.15.7:0/3826705311 shutdown_connections 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.623+0000 7fa89e0c3700 1 -- 172.21.15.7:0/3826705311 wait complete. 2023-10-06T11:22:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 Processor -- start 2023-10-06T11:22:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- start start 2023-10-06T11:22:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa8980aac00 0x7fa8981520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa89815cb70 0x7fa8980697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89814da40 con 0x7fa8980ae990 2023-10-06T11:22:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8981525e0 con 0x7fa89815cb70 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898150480 con 0x7fa8980aac00 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39936/0 (socket says 172.21.15.7:39936) 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 -- 172.21.15.7:0/2521924424 learned_addr learned my addr 172.21.15.7:0/2521924424 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa8980aac00 0x7fa8981520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89d8c2700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa89815cb70 0x7fa8980697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 -- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa8980aac00 msgr2=0x7fa8981520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa8980aac00 0x7fa8981520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 -- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa89815cb70 msgr2=0x7fa8980697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa89815cb70 0x7fa8980697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 -- 172.21.15.7:0/2521924424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009420 con 0x7fa8980ae990 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89c8c0700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa89400ec80 tx=0x7fa8940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400c920 con 0x7fa8980ae990 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89400ca80 con 0x7fa8980ae990 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa888009420 con 0x7fa8980ae990 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400ce00 con 0x7fa8980ae990 2023-10-06T11:22:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.627+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa898150a40 con 0x7fa8980ae990 2023-10-06T11:22:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89806c380 con 0x7fa8980ae990 2023-10-06T11:22:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa894013680 con 0x7fa8980ae990 2023-10-06T11:22:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa88d7fa700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 0x7fa884062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa89407cfa0 con 0x7fa8980ae990 2023-10-06T11:22:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 0x7fa884062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.631+0000 7fa89d0c1700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 0x7fa884062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8880091e0 tx=0x7fa888017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.639+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa8940acbe0 con 0x7fa8980ae990 2023-10-06T11:22:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.803+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa89804b5c0 con 0x7fa8980ae990 2023-10-06T11:22:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.803+0000 7fa88d7fa700 1 -- 172.21.15.7:0/2521924424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa89404d140 con 0x7fa8980ae990 2023-10-06T11:22:22.810 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:22.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 msgr2=0x7fa884062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 0x7fa884062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8880091e0 tx=0x7fa888017000 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 msgr2=0x7fa89814d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa89400ec80 tx=0x7fa8940091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 shutdown_connections 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa884060550 0x7fa884062a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa89815cb70 0x7fa8980697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8980ae990 0x7fa89814d150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 --2- 172.21.15.7:0/2521924424 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa8980aac00 0x7fa8981520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 >> 172.21.15.7:0/2521924424 conn(0x7fa8980cc3c0 msgr2=0x7fa8980a9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 shutdown_connections 2023-10-06T11:22:22.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.811+0000 7fa89e0c3700 1 -- 172.21.15.7:0/2521924424 wait complete. 2023-10-06T11:22:22.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:22.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 Processor -- start 2023-10-06T11:22:22.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 -- start start 2023-10-06T11:22:22.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86ec06c2e0 0x7f86ec0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0c8be0 con 0x7f86ec15c8a0 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec06aa70 con 0x7f86ec06c2e0 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f3a0b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec06ad50 con 0x7f86ec15c4d0 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86ec06c2e0 0x7f86ec0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86ec06c2e0 0x7f86ec0c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39938/0 (socket says 172.21.15.7:39938) 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 -- 172.21.15.7:0/200757005 learned_addr learned my addr 172.21.15.7:0/200757005 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f17a7700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 -- 172.21.15.7:0/200757005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 msgr2=0x7f86ec069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 -- 172.21.15.7:0/200757005 >> v1:172.21.15.50:6789/0 conn(0x7f86ec15c8a0 legacy=0x7f86ec06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f1fa8700 1 -- 172.21.15.7:0/200757005 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0c8870 con 0x7f86ec06c2e0 2023-10-06T11:22:22.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.943+0000 7f86f17a7700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f1fa8700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec06c2e0 0x7f86ec0c9070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009f20 tx=0x7f86e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395e4db9d70963e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86e3fff700 1 -- 172.21.15.7:0/200757005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400e070 con 0x7f86ec06c2e0 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86e3fff700 1 -- 172.21.15.7:0/200757005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e400ba80 con 0x7f86ec06c2e0 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86e3fff700 1 -- 172.21.15.7:0/200757005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40092e0 con 0x7f86ec06c2e0 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- 172.21.15.7:0/200757005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec06c2e0 msgr2=0x7f86ec0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec06c2e0 0x7f86ec0c9070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009f20 tx=0x7f86e4009840 comp rx=0 tx=0).stop 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- 172.21.15.7:0/200757005 shutdown_connections 2023-10-06T11:22:22.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec06c2e0 0x7f86ec0c9070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/200757005 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- 172.21.15.7:0/200757005 >> 172.21.15.7:0/200757005 conn(0x7f86ec0cdc90 msgr2=0x7f86ec0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- 172.21.15.7:0/200757005 shutdown_connections 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- 172.21.15.7:0/200757005 wait complete. 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 Processor -- start 2023-10-06T11:22:22.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.947+0000 7f86f3a0b700 1 -- start start 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86ec06c2e0 0x7f86ec169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0b1420 con 0x7f86ec15c8a0 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec169b80 con 0x7f86ec06c2e0 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0b1670 con 0x7f86ec15c4d0 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39948/0 (socket says 172.21.15.7:39948) 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 -- 172.21.15.7:0/3533656427 learned_addr learned my addr 172.21.15.7:0/3533656427 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f17a7700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86ec06c2e0 0x7f86ec169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f0fa6700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 -- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 msgr2=0x7f86ec0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 -- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86ec06c2e0 msgr2=0x7f86ec169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86ec06c2e0 0x7f86ec169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 -- 172.21.15.7:0/3533656427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009ae0 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f1fa8700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86ec06e9e0 tx=0x7f86e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40032f0 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4008d20 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec0b1950 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4026690 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec0b36d0 con 0x7f86ec15c8a0 2023-10-06T11:22:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec0b3890 con 0x7f86ec15c8a0 2023-10-06T11:22:22.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.951+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f86e4003040 con 0x7f86ec15c8a0 2023-10-06T11:22:22.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.955+0000 7f86e17fa700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 0x7f86d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:22.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.955+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f86e407ea70 con 0x7f86ec15c8a0 2023-10-06T11:22:22.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.955+0000 7f86f17a7700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 0x7f86d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:22.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.955+0000 7f86f17a7700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 0x7f86d0062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006f00 tx=0x7f86dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:22.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:22.963+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f86ec0b3890 con 0x7f86ec15c8a0 2023-10-06T11:22:23.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.119+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86ec0b3890 con 0x7f86ec15c8a0 2023-10-06T11:22:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.119+0000 7f86e17fa700 1 -- 172.21.15.7:0/3533656427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86ec0b3890 con 0x7f86ec15c8a0 2023-10-06T11:22:23.126 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 msgr2=0x7f86d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 0x7f86d0062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006f00 tx=0x7f86dc008040 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 msgr2=0x7f86ec16b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86ec06e9e0 tx=0x7f86e4018000 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 shutdown_connections 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f86d0060550 0x7f86d0062a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ec15c8a0 0x7f86ec16b570 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f86ec15c4d0 0x7f86ec0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 --2- 172.21.15.7:0/3533656427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86ec06c2e0 0x7f86ec169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 >> 172.21.15.7:0/3533656427 conn(0x7f86ec0cdc90 msgr2=0x7f86ec06a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 shutdown_connections 2023-10-06T11:22:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.123+0000 7f86f3a0b700 1 -- 172.21.15.7:0/3533656427 wait complete. 2023-10-06T11:22:23.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.247+0000 7f9c3234d700 1 Processor -- start 2023-10-06T11:22:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.247+0000 7f9c3234d700 1 -- start start 2023-10-06T11:22:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3234d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c2c156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3234d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c1571f0 con 0x7f9c2c1569a0 2023-10-06T11:22:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3234d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c0adcd0 con 0x7f9c2c158f30 2023-10-06T11:22:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3234d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c157b00 con 0x7f9c2c156090 2023-10-06T11:22:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c2c1569a0 0x7f9c2c0acf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35026/0 (socket says 172.21.15.7:35026) 2023-10-06T11:22:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 -- 172.21.15.7:0/1411257794 learned_addr learned my addr 172.21.15.7:0/1411257794 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3134b700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c2c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483703010 0 0) 0x7f9c2c1571f0 con 0x7f9c2c1569a0 2023-10-06T11:22:23.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c10003570 con 0x7f9c2c1569a0 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716012742 0 0) 0x7f9c2c0adcd0 con 0x7f9c2c158f30 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c2c1571f0 con 0x7f9c2c158f30 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414260533 0 0) 0x7f9c10003570 con 0x7f9c2c1569a0 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 msgr2=0x7f9c2c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c2c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 >> v1:172.21.15.50:6789/0 conn(0x7f9c2c158f30 legacy=0x7f9c2c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c157440 con 0x7f9c2c1569a0 2023-10-06T11:22:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f9c18001470 con 0x7f9c2c1569a0 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 >> v1:172.21.15.7:6789/0 conn(0x7f9c2c1569a0 legacy=0x7f9c2c0acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c10003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10004180 0x7f9c10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c10004d50 0x7f9c100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c157b00 con 0x7f9c10004180 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1000bba0 con 0x7f9c10004d50 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1000bd60 con 0x7f9c2c156090 2023-10-06T11:22:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10004180 0x7f9c10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c3134b700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c10003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c31b4c700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c10004d50 0x7f9c100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 -- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 msgr2=0x7f9c10003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c156090 0x7f9c10003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 -- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c10004d50 msgr2=0x7f9c100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c10004d50 0x7f9c100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 -- 172.21.15.7:0/1411257794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10003570 con 0x7f9c10004180 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10004180 0x7f9c10004810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c2d0 tx=0x7f9c2c157a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb0a8ebef84fdbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800cba0 con 0x7f9c10004180 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1800cd00 con 0x7f9c10004180 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.251+0000 7f9c2bfff700 1 -- 172.21.15.7:0/1411257794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18005b70 con 0x7f9c10004180 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- 172.21.15.7:0/1411257794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10004180 msgr2=0x7f9c10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1411257794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10004180 0x7f9c10004810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c2d0 tx=0x7f9c2c157a00 comp rx=0 tx=0).stop 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c31b4c700 1 -- 172.21.15.7:0/1411257794 reap_dead start 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- 172.21.15.7:0/1411257794 shutdown_connections 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- 172.21.15.7:0/1411257794 >> 172.21.15.7:0/1411257794 conn(0x7f9c2c0cc3c0 msgr2=0x7f9c2c067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- 172.21.15.7:0/1411257794 shutdown_connections 2023-10-06T11:22:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- 172.21.15.7:0/1411257794 wait complete. 2023-10-06T11:22:23.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 Processor -- start 2023-10-06T11:22:23.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- start start 2023-10-06T11:22:23.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c2c158370 0x7f9c2c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c0ab2f0 0x7f9c2c0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c158d20 con 0x7f9c2c143960 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0aa4f0 con 0x7f9c2c158370 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3234d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0aa060 con 0x7f9c2c0ab2f0 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39978/0 (socket says 172.21.15.7:39978) 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 -- 172.21.15.7:0/1340912825 learned_addr learned my addr 172.21.15.7:0/1340912825 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c31b4c700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c0ab2f0 0x7f9c2c0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c2c158370 0x7f9c2c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 -- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c0ab2f0 msgr2=0x7f9c2c0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c0ab2f0 0x7f9c2c0ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 -- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c2c158370 msgr2=0x7f9c2c1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c2c158370 0x7f9c2c1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 -- 172.21.15.7:0/1340912825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0a91b0 con 0x7f9c2c143960 2023-10-06T11:22:23.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.255+0000 7f9c3134b700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c011fb0 tx=0x7f9c1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00e040 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00f040 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c0a9450 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c015430 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c0a9a10 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c06c380 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.259+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c1c0157e0 con 0x7f9c2c143960 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.263+0000 7f9c297fa700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 0x7f9c14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.263+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9c1c07d830 con 0x7f9c2c143960 2023-10-06T11:22:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.263+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 0x7f9c14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.263+0000 7f9c30b4a700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 0x7f9c14062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c2d0 tx=0x7f9c18015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.267+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c1c04d930 con 0x7f9c2c143960 2023-10-06T11:22:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c2c157720 con 0x7f9c2c143960 2023-10-06T11:22:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c297fa700 1 -- 172.21.15.7:0/1340912825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c1c050f50 con 0x7f9c2c143960 2023-10-06T11:22:23.435 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 msgr2=0x7f9c14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 0x7f9c14062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c2d0 tx=0x7f9c18015000 comp rx=0 tx=0).stop 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 msgr2=0x7f9c2c169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c011fb0 tx=0x7f9c1c00b040 comp rx=0 tx=0).stop 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 shutdown_connections 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9c14060550 0x7f9c14062a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9c2c0ab2f0 0x7f9c2c0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c2c158370 0x7f9c2c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 --2- 172.21.15.7:0/1340912825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2c143960 0x7f9c2c169e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 >> 172.21.15.7:0/1340912825 conn(0x7f9c2c0cc3c0 msgr2=0x7f9c2c0c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 shutdown_connections 2023-10-06T11:22:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.431+0000 7f9c3234d700 1 -- 172.21.15.7:0/1340912825 wait complete. 2023-10-06T11:22:23.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:23.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 Processor -- start 2023-10-06T11:22:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 -- start start 2023-10-06T11:22:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37b80b0b70 0x7f37b80ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b80ac8e0 con 0x7f37b80aaee0 2023-10-06T11:22:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b8156110 con 0x7f37b80b0b70 2023-10-06T11:22:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37bce73700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80ab2b0 con 0x7f37b80cc1d0 2023-10-06T11:22:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f37b80aaee0 0x7f37b80b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35032/0 (socket says 172.21.15.7:35032) 2023-10-06T11:22:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b659c700 1 -- 172.21.15.7:0/862298413 learned_addr learned my addr 172.21.15.7:0/862298413 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/862298413 >> v2:172.21.15.7:3300/0 conn(0x7f37b80b0b70 0x7f37b80ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b6d9d700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 -- 172.21.15.7:0/862298413 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 msgr2=0x7f37b80c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 -- 172.21.15.7:0/862298413 >> v1:172.21.15.7:6789/0 conn(0x7f37b80aaee0 legacy=0x7f37b80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 -- 172.21.15.7:0/862298413 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b80acbc0 con 0x7f37b80b0b70 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80ad980 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0098e0 tx=0x7f37ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff34a575b83a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.563+0000 7f37b559a700 1 -- 172.21.15.7:0/862298413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00f6b0 con 0x7f37b80b0b70 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b559a700 1 -- 172.21.15.7:0/862298413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00f810 con 0x7f37b80b0b70 2023-10-06T11:22:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b559a700 1 -- 172.21.15.7:0/862298413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00fbb0 con 0x7f37b80b0b70 2023-10-06T11:22:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- 172.21.15.7:0/862298413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 msgr2=0x7f37b80ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80ad980 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0098e0 tx=0x7f37ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- 172.21.15.7:0/862298413 shutdown_connections 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- 172.21.15.7:0/862298413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80ad980 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- 172.21.15.7:0/862298413 >> 172.21.15.7:0/862298413 conn(0x7f37b8068900 msgr2=0x7f37b80cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- 172.21.15.7:0/862298413 shutdown_connections 2023-10-06T11:22:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- 172.21.15.7:0/862298413 wait complete. 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 Processor -- start 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- start start 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 0x7f37b806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80aa330 con 0x7f37b80b0b70 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80a9290 con 0x7f37b80aaee0 2023-10-06T11:22:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37bce73700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80a9570 con 0x7f37b80cc1d0 2023-10-06T11:22:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40000/0 (socket says 172.21.15.7:40000) 2023-10-06T11:22:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 -- 172.21.15.7:0/3542271918 learned_addr learned my addr 172.21.15.7:0/3542271918 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b659c700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 0x7f37b806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b6d9d700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 -- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 msgr2=0x7f37b80c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 -- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 msgr2=0x7f37b806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.567+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 0x7f37b806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37b5d9b700 1 -- 172.21.15.7:0/3542271918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009420 con 0x7f37b80b0b70 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37b6d9d700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37b659c700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 0x7f37b806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37b5d9b700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0093f0 tx=0x7f37ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01ad80 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac010470 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b80a9850 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac010830 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b80a9e10 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f37ac0183e0 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b80aa030 con 0x7f37b80b0b70 2023-10-06T11:22:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 0x7f3798062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f37ac014070 con 0x7f37b80b0b70 2023-10-06T11:22:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.571+0000 7f37b659c700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 0x7f3798062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.575+0000 7f37b659c700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 0x7f3798062a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f37b80b05f0 tx=0x7f37a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.579+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f37ac04eb60 con 0x7f37b80b0b70 2023-10-06T11:22:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.739+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f37b804be00 con 0x7f37b80b0b70 2023-10-06T11:22:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.739+0000 7f37a67fc700 1 -- 172.21.15.7:0/3542271918 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f37ac052180 con 0x7f37b80b0b70 2023-10-06T11:22:23.745 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 msgr2=0x7f3798062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 0x7f3798062a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f37b80b05f0 tx=0x7f37a8009040 comp rx=0 tx=0).stop 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 msgr2=0x7f37b80a88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0093f0 tx=0x7f37ac019000 comp rx=0 tx=0).stop 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 shutdown_connections 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f37980605d0 0x7f3798062a80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f37b80cc1d0 0x7f37b80c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b80b0b70 0x7f37b80a88b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 --2- 172.21.15.7:0/3542271918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b80aaee0 0x7f37b806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 >> 172.21.15.7:0/3542271918 conn(0x7f37b8068900 msgr2=0x7f37b8154820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 shutdown_connections 2023-10-06T11:22:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.743+0000 7f37bce73700 1 -- 172.21.15.7:0/3542271918 wait complete. 2023-10-06T11:22:23.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:23.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 Processor -- start 2023-10-06T11:22:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 -- start start 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce0053260 con 0x7f6ce00500a0 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce0006e60 con 0x7f6ce00e72f0 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cef46d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00504f0 con 0x7f6ce0033c50 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ceec6c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:55148/0 (socket says 172.21.15.7:55148) 2023-10-06T11:22:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 -- 172.21.15.7:0/650732313 learned_addr learned my addr 172.21.15.7:0/650732313 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cedc6a700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f6ce00500a0 0x7f6ce00e6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:40506/0 (socket says 172.21.15.7:40506) 2023-10-06T11:22:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ceec6c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:36230/0 (socket says 172.21.15.7:36230) 2023-10-06T11:22:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 -- 172.21.15.7:0/650732313 >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 msgr2=0x7f6ce0006920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 --2- 172.21.15.7:0/650732313 >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 -- 172.21.15.7:0/650732313 >> v1:172.21.15.50:6789/0 conn(0x7f6ce00500a0 legacy=0x7f6ce00e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 -- 172.21.15.7:0/650732313 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00070c0 con 0x7f6ce0033c50 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ceec6c700 1 --2- 172.21.15.7:0/650732313 >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6cee46b700 1 --2- 172.21.15.7:0/650732313 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0099b0 tx=0x7f6cdc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529fcf4729449adf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ced469700 1 -- 172.21.15.7:0/650732313 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc009ea0 con 0x7f6ce0033c50 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ced469700 1 -- 172.21.15.7:0/650732313 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00b860 con 0x7f6ce0033c50 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.863+0000 7f6ced469700 1 -- 172.21.15.7:0/650732313 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00bc00 con 0x7f6ce0033c50 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- 172.21.15.7:0/650732313 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 msgr2=0x7f6ce0034020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- 172.21.15.7:0/650732313 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0099b0 tx=0x7f6cdc009250 comp rx=0 tx=0).stop 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- 172.21.15.7:0/650732313 shutdown_connections 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- 172.21.15.7:0/650732313 >> v2:172.21.15.50:3300/0 conn(0x7f6ce00e72f0 0x7f6ce0006920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- 172.21.15.7:0/650732313 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce0033c50 0x7f6ce0034020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- 172.21.15.7:0/650732313 >> 172.21.15.7:0/650732313 conn(0x7f6ce0054d40 msgr2=0x7f6ce0055110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- 172.21.15.7:0/650732313 shutdown_connections 2023-10-06T11:22:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- 172.21.15.7:0/650732313 wait complete. 2023-10-06T11:22:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 Processor -- start 2023-10-06T11:22:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- start start 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ce0033c50 0x7f6ce00e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce00e72f0 0x7f6ce00df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce004bdf0 con 0x7f6ce00500a0 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce004c970 con 0x7f6ce0033c50 2023-10-06T11:22:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.867+0000 7f6cef46d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce004e9f0 con 0x7f6ce00e72f0 2023-10-06T11:22:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40002/0 (socket says 172.21.15.7:40002) 2023-10-06T11:22:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 -- 172.21.15.7:0/500577778 learned_addr learned my addr 172.21.15.7:0/500577778 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6ceec6c700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce00e72f0 0x7f6ce00df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cee46b700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ce0033c50 0x7f6ce00e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 -- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce00e72f0 msgr2=0x7f6ce00df9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce00e72f0 0x7f6ce00df9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 -- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ce0033c50 msgr2=0x7f6ce00e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ce0033c50 0x7f6ce00e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 -- 172.21.15.7:0/500577778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009420 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cedc6a700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ce806fa50 tx=0x7f6ce806fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce804bf00 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce806b140 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0094f0 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce806b4c0 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce004c320 con 0x7f6ce00500a0 2023-10-06T11:22:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce004cae0 con 0x7f6ce00500a0 2023-10-06T11:22:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.871+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6ce806bab0 con 0x7f6ce00500a0 2023-10-06T11:22:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.875+0000 7f6cda7fc700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 0x7f6ccc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.875+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6ce8068e40 con 0x7f6ce00500a0 2023-10-06T11:22:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.875+0000 7f6cee46b700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 0x7f6ccc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.875+0000 7f6cee46b700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 0x7f6ccc062a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0099b0 tx=0x7f6cdc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:23.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:23.879+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ce80c31d0 con 0x7f6ce00500a0 2023-10-06T11:22:24.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.075+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ce00053f0 con 0x7f6ce00500a0 2023-10-06T11:22:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.075+0000 7f6cda7fc700 1 -- 172.21.15.7:0/500577778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f6ce80cae60 con 0x7f6ce00500a0 2023-10-06T11:22:24.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:24.081 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:24.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 msgr2=0x7f6ccc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 0x7f6ccc062a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0099b0 tx=0x7f6cdc017000 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 msgr2=0x7f6ce00e2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ce806fa50 tx=0x7f6ce806fd00 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 shutdown_connections 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6ccc060580 0x7f6ccc062a30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ce00e72f0 0x7f6ce00df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ce00500a0 0x7f6ce00e2f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 --2- 172.21.15.7:0/500577778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ce0033c50 0x7f6ce00e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 >> 172.21.15.7:0/500577778 conn(0x7f6ce0054d40 msgr2=0x7f6ce004b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 shutdown_connections 2023-10-06T11:22:24.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.079+0000 7f6cef46d700 1 -- 172.21.15.7:0/500577778 wait complete. 2023-10-06T11:22:24.085 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:24.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:24.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 Processor -- start 2023-10-06T11:22:24.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 -- start start 2023-10-06T11:22:24.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 0x7f99a00cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99a0155c10 0x7f99a00ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a00c8c20 con 0x7f99a0155840 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a015a610 con 0x7f99a0155c10 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c8610 con 0x7f99a006c0b0 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99a0155840 0x7f99a0159670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35048/0 (socket says 172.21.15.7:35048) 2023-10-06T11:22:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999ffff700 1 -- 172.21.15.7:0/2083042061 learned_addr learned my addr 172.21.15.7:0/2083042061 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a4c0f700 1 --2- 172.21.15.7:0/2083042061 >> v2:172.21.15.7:3300/0 conn(0x7f99a0155c10 0x7f99a00ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 --2- 172.21.15.7:0/2083042061 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 0x7f99a00cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999effd700 1 -- 172.21.15.7:0/2083042061 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790580459 0 0) 0x7f99a00c8c20 con 0x7f99a0155840 2023-10-06T11:22:24.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999effd700 1 -- 172.21.15.7:0/2083042061 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9988003570 con 0x7f99a0155840 2023-10-06T11:22:24.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 -- 172.21.15.7:0/2083042061 >> v2:172.21.15.7:3300/0 conn(0x7f99a0155c10 msgr2=0x7f99a00ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:24.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 --2- 172.21.15.7:0/2083042061 >> v2:172.21.15.7:3300/0 conn(0x7f99a0155c10 0x7f99a00ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 -- 172.21.15.7:0/2083042061 >> v1:172.21.15.7:6789/0 conn(0x7f99a0155840 legacy=0x7f99a0159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 -- 172.21.15.7:0/2083042061 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0159d80 con 0x7f99a006c0b0 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999f7fe700 1 --2- 172.21.15.7:0/2083042061 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 0x7f99a00cc9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99940098e0 tx=0x7f99940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360ce144b70027b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999effd700 1 -- 172.21.15.7:0/2083042061 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994011630 con 0x7f99a006c0b0 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999effd700 1 -- 172.21.15.7:0/2083042061 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994011790 con 0x7f99a006c0b0 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f999effd700 1 -- 172.21.15.7:0/2083042061 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994011b30 con 0x7f99a006c0b0 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 -- 172.21.15.7:0/2083042061 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 msgr2=0x7f99a00cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.199+0000 7f99a5410700 1 --2- 172.21.15.7:0/2083042061 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 0x7f99a00cc9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99940098e0 tx=0x7f99940091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/2083042061 shutdown_connections 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 --2- 172.21.15.7:0/2083042061 >> v2:172.21.15.7:3300/0 conn(0x7f99a0155c10 0x7f99a00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 --2- 172.21.15.7:0/2083042061 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a006c0b0 0x7f99a00cc9c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/2083042061 >> 172.21.15.7:0/2083042061 conn(0x7f99a006f380 msgr2=0x7f99a006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/2083042061 shutdown_connections 2023-10-06T11:22:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/2083042061 wait complete. 2023-10-06T11:22:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 Processor -- start 2023-10-06T11:22:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- start start 2023-10-06T11:22:24.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a0155840 0x7f99a00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a0155c10 0x7f99a0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01592d0 con 0x7f99a0156a80 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c94f0 con 0x7f99a0155840 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0157b20 con 0x7f99a0155c10 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 -- 172.21.15.7:0/560091055 learned_addr learned my addr 172.21.15.7:0/560091055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f999f7fe700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a0155c10 0x7f99a0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f999ffff700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a0155840 0x7f99a00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 -- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a0155c10 msgr2=0x7f99a0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a0155c10 0x7f99a0157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 -- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a0155840 msgr2=0x7f99a00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a0155840 0x7f99a00c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 -- 172.21.15.7:0/560091055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009420 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a4c0f700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9998009e90 tx=0x7f999800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800cb10 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800f040 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a0156f10 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800cee0 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.203+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a01574d0 con 0x7f99a0156a80 2023-10-06T11:22:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.207+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0157c90 con 0x7f99a0156a80 2023-10-06T11:22:24.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.211+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9998014690 con 0x7f99a0156a80 2023-10-06T11:22:24.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.211+0000 7f9987fff700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 0x7f998c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.211+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f999807dda0 con 0x7f99a0156a80 2023-10-06T11:22:24.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.211+0000 7f999ffff700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 0x7f998c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.211+0000 7f999ffff700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 0x7f998c062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9990007fd0 tx=0x7f9990009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.215+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99980519b0 con 0x7f99a0156a80 2023-10-06T11:22:24.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.383+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a004b5c0 con 0x7f99a0156a80 2023-10-06T11:22:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.383+0000 7f9987fff700 1 -- 172.21.15.7:0/560091055 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f9998051540 con 0x7f99a0156a80 2023-10-06T11:22:24.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:24.389 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:24.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 msgr2=0x7f998c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 0x7f998c062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9990007fd0 tx=0x7f9990009040 comp rx=0 tx=0).stop 2023-10-06T11:22:24.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 msgr2=0x7f99a00caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9998009e90 tx=0x7f999800b040 comp rx=0 tx=0).stop 2023-10-06T11:22:24.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 shutdown_connections 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f998c060550 0x7f998c062a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99a0156a80 0x7f99a00caa30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99a0155c10 0x7f99a0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 --2- 172.21.15.7:0/560091055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a0155840 0x7f99a00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 >> 172.21.15.7:0/560091055 conn(0x7f99a006f380 msgr2=0x7f99a0154830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 shutdown_connections 2023-10-06T11:22:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.387+0000 7f99a5410700 1 -- 172.21.15.7:0/560091055 wait complete. 2023-10-06T11:22:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:24.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.503+0000 7fd7258c2700 1 Processor -- start 2023-10-06T11:22:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7258c2700 1 -- start start 2023-10-06T11:22:24.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7258c2700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd7200ccdb0 0x7fd7200ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7258c2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72015d320 con 0x7fd720159f20 2023-10-06T11:22:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7258c2700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200ce940 con 0x7fd7200b21d0 2023-10-06T11:22:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7258c2700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72015c9f0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7250c1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd720159f20 0x7fd7200ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35054/0 (socket says 172.21.15.7:35054) 2023-10-06T11:22:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7250c1700 1 -- 172.21.15.7:0/3098456866 learned_addr learned my addr 172.21.15.7:0/3098456866 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd717fff700 1 --2- 172.21.15.7:0/3098456866 >> v2:172.21.15.50:3300/0 conn(0x7fd7200ccdb0 0x7fd7200ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503450447 0 0) 0x7fd72015d320 con 0x7fd720159f20 2023-10-06T11:22:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd708003570 con 0x7fd720159f20 2023-10-06T11:22:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060847507 0 0) 0x7fd7200ce940 con 0x7fd7200b21d0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72015d320 con 0x7fd7200b21d0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd717fff700 1 -- 172.21.15.7:0/3098456866 >> v1:172.21.15.50:6789/0 conn(0x7fd7200b21d0 legacy=0x7fd7200cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194520637 0 0) 0x7fd708003570 con 0x7fd720159f20 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd717fff700 1 -- 172.21.15.7:0/3098456866 >> v1:172.21.15.7:6789/0 conn(0x7fd720159f20 legacy=0x7fd7200ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd717fff700 1 -- 172.21.15.7:0/3098456866 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72015ccd0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd717fff700 1 --2- 172.21.15.7:0/3098456866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200ccdb0 0x7fd7200ce260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7100098e0 tx=0x7fd7100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59865fbfa3d84fee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180017f0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710011420 con 0x7fd7200ccdb0 2023-10-06T11:22:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.507+0000 7fd7177fe700 1 -- 172.21.15.7:0/3098456866 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100117a0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- 172.21.15.7:0/3098456866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200ccdb0 msgr2=0x7fd7200ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 --2- 172.21.15.7:0/3098456866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200ccdb0 0x7fd7200ce260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7100098e0 tx=0x7fd7100091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- 172.21.15.7:0/3098456866 shutdown_connections 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 --2- 172.21.15.7:0/3098456866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200ccdb0 0x7fd7200ce260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- 172.21.15.7:0/3098456866 >> 172.21.15.7:0/3098456866 conn(0x7fd720067e30 msgr2=0x7fd720068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- 172.21.15.7:0/3098456866 shutdown_connections 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- 172.21.15.7:0/3098456866 wait complete. 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 Processor -- start 2023-10-06T11:22:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- start start 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b21d0 0x7fd7200b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd720159f20 0x7fd720168ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200c9b90 con 0x7fd7200ccdb0 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201540d0 con 0x7fd7200b21d0 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd7258c2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200b4140 con 0x7fd720159f20 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd717fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd717fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40028/0 (socket says 172.21.15.7:40028) 2023-10-06T11:22:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.511+0000 7fd717fff700 1 -- 172.21.15.7:0/884581637 learned_addr learned my addr 172.21.15.7:0/884581637 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd7250c1700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd720159f20 0x7fd720168ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd7248c0700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b21d0 0x7fd7200b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 -- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd720159f20 msgr2=0x7fd720168ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd720159f20 0x7fd720168ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 -- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b21d0 msgr2=0x7fd7200b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b21d0 0x7fd7200b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 -- 172.21.15.7:0/884581637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009420 con 0x7fd7200ccdb0 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd717fff700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7100093f0 tx=0x7fd710016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71001cb30 con 0x7fd7200ccdb0 2023-10-06T11:22:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71001ccd0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7200b43e0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71001a6a0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7200b4770 con 0x7fd7200ccdb0 2023-10-06T11:22:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7200b4570 con 0x7fd7200ccdb0 2023-10-06T11:22:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.515+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd71001aa50 con 0x7fd7200ccdb0 2023-10-06T11:22:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.519+0000 7fd714ff9700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 0x7fd70c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.519+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd71007e3f0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.519+0000 7fd7248c0700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 0x7fd70c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.519+0000 7fd7248c0700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 0x7fd70c062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007fd0 tx=0x7fd71c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.523+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd71004e640 con 0x7fd7200ccdb0 2023-10-06T11:22:24.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.683+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7200cdec0 con 0x7fd7200ccdb0 2023-10-06T11:22:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd714ff9700 1 -- 172.21.15.7:0/884581637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd710051c60 con 0x7fd7200ccdb0 2023-10-06T11:22:24.690 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 msgr2=0x7fd70c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 0x7fd70c062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007fd0 tx=0x7fd71c007d50 comp rx=0 tx=0).stop 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 msgr2=0x7fd7200c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7100093f0 tx=0x7fd710016000 comp rx=0 tx=0).stop 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 shutdown_connections 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd70c060550 0x7fd70c062a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd720159f20 0x7fd720168ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7200ccdb0 0x7fd7200c9220 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 --2- 172.21.15.7:0/884581637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7200b21d0 0x7fd7200b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 >> 172.21.15.7:0/884581637 conn(0x7fd720067e30 msgr2=0x7fd7201587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 shutdown_connections 2023-10-06T11:22:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.687+0000 7fd7258c2700 1 -- 172.21.15.7:0/884581637 wait complete. 2023-10-06T11:22:24.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.807+0000 7fb26f59e700 1 Processor -- start 2023-10-06T11:22:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.807+0000 7fb26f59e700 1 -- start start 2023-10-06T11:22:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb270156090 0x7fb270156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2701569a0 0x7fb2700acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2701571f0 con 0x7fb2700ad4a0 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb270157b00 con 0x7fb2701569a0 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2700adcd0 con 0x7fb270156090 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2701569a0 0x7fb2700acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb2700ad4a0 0x7fb270157f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35062/0 (socket says 172.21.15.7:35062) 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2701569a0 0x7fb2700acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40044/0 (socket says 172.21.15.7:40044) 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26ed9d700 1 -- 172.21.15.7:0/3968091057 learned_addr learned my addr 172.21.15.7:0/3968091057 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26e59c700 1 --2- 172.21.15.7:0/3968091057 >> v2:172.21.15.50:3300/0 conn(0x7fb270156090 0x7fb270156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 -- 172.21.15.7:0/3968091057 >> v2:172.21.15.50:3300/0 conn(0x7fb270156090 msgr2=0x7fb270156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 --2- 172.21.15.7:0/3968091057 >> v2:172.21.15.50:3300/0 conn(0x7fb270156090 0x7fb270156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 -- 172.21.15.7:0/3968091057 >> v1:172.21.15.7:6789/0 conn(0x7fb2700ad4a0 legacy=0x7fb270157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 -- 172.21.15.7:0/3968091057 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270157440 con 0x7fb2701569a0 2023-10-06T11:22:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb265d9b700 1 --2- 172.21.15.7:0/3968091057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2701569a0 0x7fb2700acf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb27006d500 tx=0x7fb2580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802c9984df757cd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26dd9b700 1 -- 172.21.15.7:0/3968091057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800b6b0 con 0x7fb2701569a0 2023-10-06T11:22:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26dd9b700 1 -- 172.21.15.7:0/3968091057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25800b810 con 0x7fb2701569a0 2023-10-06T11:22:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26dd9b700 1 -- 172.21.15.7:0/3968091057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800bbb0 con 0x7fb2701569a0 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- 172.21.15.7:0/3968091057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2701569a0 msgr2=0x7fb2700acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 --2- 172.21.15.7:0/3968091057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2701569a0 0x7fb2700acf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb27006d500 tx=0x7fb2580091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- 172.21.15.7:0/3968091057 shutdown_connections 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 --2- 172.21.15.7:0/3968091057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2701569a0 0x7fb2700acf60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 --2- 172.21.15.7:0/3968091057 >> v2:172.21.15.50:3300/0 conn(0x7fb270156090 0x7fb270156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- 172.21.15.7:0/3968091057 >> 172.21.15.7:0/3968091057 conn(0x7fb2700cc3c0 msgr2=0x7fb270067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- 172.21.15.7:0/3968091057 shutdown_connections 2023-10-06T11:22:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.811+0000 7fb26f59e700 1 -- 172.21.15.7:0/3968091057 wait complete. 2023-10-06T11:22:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 Processor -- start 2023-10-06T11:22:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- start start 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb270156090 0x7fb27014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2701569a0 0x7fb2700ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270145d90 con 0x7fb2700ad4a0 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270144f10 con 0x7fb270156090 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270146bc0 con 0x7fb2701569a0 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40050/0 (socket says 172.21.15.7:40050) 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 -- 172.21.15.7:0/2913918411 learned_addr learned my addr 172.21.15.7:0/2913918411 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26ed9d700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2701569a0 0x7fb2700ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb265d9b700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb270156090 0x7fb27014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 -- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2701569a0 msgr2=0x7fb2700ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2701569a0 0x7fb2700ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 -- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb270156090 msgr2=0x7fb27014b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb270156090 0x7fb27014b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 -- 172.21.15.7:0/2913918411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258009420 con 0x7fb2700ad4a0 2023-10-06T11:22:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26e59c700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb26000d8c0 tx=0x7fb260008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600125f0 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260012750 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260012af0 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb270146220 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.815+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb270145460 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27006c380 con 0x7fb2700ad4a0 2023-10-06T11:22:24.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb26001e460 con 0x7fb2700ad4a0 2023-10-06T11:22:24.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb266ffd700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 0x7fb250062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:24.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb26007d220 con 0x7fb2700ad4a0 2023-10-06T11:22:24.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb265d9b700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 0x7fb250062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:24.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.819+0000 7fb265d9b700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 0x7fb250062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2580091e0 tx=0x7fb258017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:24.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.827+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb26004d320 con 0x7fb2700ad4a0 2023-10-06T11:22:24.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.987+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb270157720 con 0x7fb2700ad4a0 2023-10-06T11:22:24.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.987+0000 7fb266ffd700 1 -- 172.21.15.7:0/2913918411 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb260050940 con 0x7fb2700ad4a0 2023-10-06T11:22:24.993 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 msgr2=0x7fb250062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 0x7fb250062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2580091e0 tx=0x7fb258017000 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 msgr2=0x7fb2700b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb26000d8c0 tx=0x7fb260008040 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 shutdown_connections 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb250060550 0x7fb250062a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2701569a0 0x7fb2700ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb270156090 0x7fb27014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 --2- 172.21.15.7:0/2913918411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2700ad4a0 0x7fb2700b04d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 >> 172.21.15.7:0/2913918411 conn(0x7fb2700cc3c0 msgr2=0x7fb270069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 shutdown_connections 2023-10-06T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:24.991+0000 7fb26f59e700 1 -- 172.21.15.7:0/2913918411 wait complete. 2023-10-06T11:22:25.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.119+0000 7fe880464700 1 Processor -- start 2023-10-06T11:22:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.119+0000 7fe880464700 1 -- start start 2023-10-06T11:22:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe878155d10 0x7fe8780c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe878156620 con 0x7fe87806e110 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe878159ff0 con 0x7fe878155940 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87815a2d0 con 0x7fe878155d10 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe878155d10 0x7fe8780c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d9ff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe87806e110 0x7fe878156ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35076/0 (socket says 172.21.15.7:35076) 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d9ff700 1 -- 172.21.15.7:0/351665299 learned_addr learned my addr 172.21.15.7:0/351665299 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe878155d10 0x7fe8780c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40066/0 (socket says 172.21.15.7:40066) 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 -- 172.21.15.7:0/351665299 >> v1:172.21.15.50:6789/0 conn(0x7fe878155940 legacy=0x7fe8781597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 -- 172.21.15.7:0/351665299 >> v1:172.21.15.7:6789/0 conn(0x7fe87806e110 legacy=0x7fe878156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d1fe700 1 -- 172.21.15.7:0/351665299 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565086694 0 0) 0x7fe878159ff0 con 0x7fe878155940 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 -- 172.21.15.7:0/351665299 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87815a5b0 con 0x7fe878155d10 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87ea01700 1 --2- 172.21.15.7:0/351665299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c7f10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d0d0 tx=0x7fe87000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b835438c823b0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d1fe700 1 -- 172.21.15.7:0/351665299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870010070 con 0x7fe878155d10 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d1fe700 1 -- 172.21.15.7:0/351665299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000ddf0 con 0x7fe878155d10 2023-10-06T11:22:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe87d1fe700 1 -- 172.21.15.7:0/351665299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b320 con 0x7fe878155d10 2023-10-06T11:22:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- 172.21.15.7:0/351665299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 msgr2=0x7fe8780c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 --2- 172.21.15.7:0/351665299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c7f10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d0d0 tx=0x7fe87000b990 comp rx=0 tx=0).stop 2023-10-06T11:22:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- 172.21.15.7:0/351665299 shutdown_connections 2023-10-06T11:22:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 --2- 172.21.15.7:0/351665299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c7f10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.123+0000 7fe880464700 1 -- 172.21.15.7:0/351665299 >> 172.21.15.7:0/351665299 conn(0x7fe8780cc2f0 msgr2=0x7fe8780cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- 172.21.15.7:0/351665299 shutdown_connections 2023-10-06T11:22:25.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- 172.21.15.7:0/351665299 wait complete. 2023-10-06T11:22:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 Processor -- start 2023-10-06T11:22:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- start start 2023-10-06T11:22:25.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe87806e110 0x7fe8781433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 0x7fe8780c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87814d2f0 con 0x7fe878155d10 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780c6040 con 0x7fe878155940 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe880464700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780cb820 con 0x7fe87806e110 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe87ea01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe87ea01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40070/0 (socket says 172.21.15.7:40070) 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe87ea01700 1 -- 172.21.15.7:0/4294225299 learned_addr learned my addr 172.21.15.7:0/4294225299 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe87d9ff700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 0x7fe8780c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.127+0000 7fe87e200700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe87806e110 0x7fe8781433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 -- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe87806e110 msgr2=0x7fe8781433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe87806e110 0x7fe8781433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 -- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 msgr2=0x7fe8780c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 0x7fe8780c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 -- 172.21.15.7:0/4294225299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87000bc30 con 0x7fe878155d10 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87d9ff700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 0x7fe8780c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe87ea01700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe878063620 tx=0x7fe87001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870009c50 con 0x7fe878155d10 2023-10-06T11:22:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870023070 con 0x7fe878155d10 2023-10-06T11:22:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87814c9c0 con 0x7fe878155d10 2023-10-06T11:22:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870028430 con 0x7fe878155d10 2023-10-06T11:22:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87814cf80 con 0x7fe878155d10 2023-10-06T11:22:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.131+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87814d4a0 con 0x7fe878155d10 2023-10-06T11:22:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.135+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe870025030 con 0x7fe878155d10 2023-10-06T11:22:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.135+0000 7fe86e7fc700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 0x7fe85c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.135+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe870080b40 con 0x7fe878155d10 2023-10-06T11:22:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.135+0000 7fe87e200700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 0x7fe85c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.135+0000 7fe87e200700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 0x7fe85c062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe874007d10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.143+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe87814d4a0 con 0x7fe878155d10 2023-10-06T11:22:25.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.303+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe87804be00 con 0x7fe878155d10 2023-10-06T11:22:25.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.303+0000 7fe86e7fc700 1 -- 172.21.15.7:0/4294225299 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe870050ce0 con 0x7fe878155d10 2023-10-06T11:22:25.310 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:25.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 msgr2=0x7fe85c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 0x7fe85c062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe874007d10 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 msgr2=0x7fe8780c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe878063620 tx=0x7fe87001a000 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 shutdown_connections 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe85c060550 0x7fe85c062a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe878155d10 0x7fe8780c5b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe878155940 0x7fe8780c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 --2- 172.21.15.7:0/4294225299 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe87806e110 0x7fe8781433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 >> 172.21.15.7:0/4294225299 conn(0x7fe8780cc2f0 msgr2=0x7fe878145020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 shutdown_connections 2023-10-06T11:22:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.307+0000 7fe880464700 1 -- 172.21.15.7:0/4294225299 wait complete. 2023-10-06T11:22:25.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.435+0000 7fdde8d9b700 1 Processor -- start 2023-10-06T11:22:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde8d9b700 1 -- start start 2023-10-06T11:22:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde8d9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 0x7fdde406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde8d9b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde415b6e0 con 0x7fdde40c8b30 2023-10-06T11:22:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde8d9b700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde406e780 con 0x7fdde415ae90 2023-10-06T11:22:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde8d9b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde415b400 con 0x7fdde4156b40 2023-10-06T11:22:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdddbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdde40c8b30 0x7fdde415a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35080/0 (socket says 172.21.15.7:35080) 2023-10-06T11:22:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdddbfff700 1 -- 172.21.15.7:0/4233075492 learned_addr learned my addr 172.21.15.7:0/4233075492 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde37fe700 1 --2- 172.21.15.7:0/4233075492 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 0x7fdde406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321613682 0 0) 0x7fdde415b6e0 con 0x7fdde40c8b30 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd0003570 con 0x7fdde40c8b30 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695632412 0 0) 0x7fdde406e780 con 0x7fdde415ae90 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde415b6e0 con 0x7fdde415ae90 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde37fe700 1 -- 172.21.15.7:0/4233075492 >> v1:172.21.15.50:6789/0 conn(0x7fdde415ae90 legacy=0x7fdde40abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde37fe700 1 -- 172.21.15.7:0/4233075492 >> v1:172.21.15.7:6789/0 conn(0x7fdde40c8b30 legacy=0x7fdde415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.439+0000 7fdde37fe700 1 -- 172.21.15.7:0/4233075492 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde406e130 con 0x7fdde4156b40 2023-10-06T11:22:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde37fe700 1 --2- 172.21.15.7:0/4233075492 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 0x7fdde406ec10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009910 tx=0x7fddd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7c624d2245e807 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011630 con 0x7fdde4156b40 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4011790 con 0x7fdde4156b40 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde2ffd700 1 -- 172.21.15.7:0/4233075492 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011b10 con 0x7fdde4156b40 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4233075492 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 msgr2=0x7fdde406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4233075492 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 0x7fdde406ec10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009910 tx=0x7fddd40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4233075492 shutdown_connections 2023-10-06T11:22:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4233075492 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde4156b40 0x7fdde406ec10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4233075492 >> 172.21.15.7:0/4233075492 conn(0x7fdde4067e30 msgr2=0x7fdde4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4233075492 shutdown_connections 2023-10-06T11:22:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.443+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4233075492 wait complete. 2023-10-06T11:22:25.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 Processor -- start 2023-10-06T11:22:25.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 -- start start 2023-10-06T11:22:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde4156b40 0x7fdde415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde415ae90 0x7fdde4154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40c4800 con 0x7fdde40c8b30 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41546c0 con 0x7fdde4156b40 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde8d9b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40c3ed0 con 0x7fdde415ae90 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40082/0 (socket says 172.21.15.7:40082) 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 -- 172.21.15.7:0/4217048486 learned_addr learned my addr 172.21.15.7:0/4217048486 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdddbfff700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde4156b40 0x7fdde415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde3fff700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde415ae90 0x7fdde4154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 -- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde415ae90 msgr2=0x7fdde4154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde415ae90 0x7fdde4154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 -- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde4156b40 msgr2=0x7fdde415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde4156b40 0x7fdde415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 -- 172.21.15.7:0/4217048486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4009420 con 0x7fdde40c8b30 2023-10-06T11:22:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.447+0000 7fdde37fe700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fddd4013000 tx=0x7fddd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd401f070 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4002f30 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde40c41b0 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011900 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde40c3bc0 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fddd4023070 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde406cf60 con 0x7fdde40c8b30 2023-10-06T11:22:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 0x7fddc4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fddd407ded0 con 0x7fdde40c8b30 2023-10-06T11:22:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddbfff700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 0x7fddc4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.451+0000 7fdddbfff700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 0x7fddc4062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdde4159fa0 tx=0x7fddcc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.455+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fddd404e050 con 0x7fdde40c8b30 2023-10-06T11:22:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.615+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdde404b5c0 con 0x7fdde40c8b30 2023-10-06T11:22:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.615+0000 7fdddb7fe700 1 -- 172.21.15.7:0/4217048486 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fddd4051630 con 0x7fdde40c8b30 2023-10-06T11:22:25.622 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 msgr2=0x7fddc4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 0x7fddc4062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdde4159fa0 tx=0x7fddcc009040 comp rx=0 tx=0).stop 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 msgr2=0x7fdde40aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fddd4013000 tx=0x7fddd4017000 comp rx=0 tx=0).stop 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 shutdown_connections 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddc4060550 0x7fddc4062a00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdde415ae90 0x7fdde4154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde4156b40 0x7fdde415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 --2- 172.21.15.7:0/4217048486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde40c8b30 0x7fdde40aaf00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 >> 172.21.15.7:0/4217048486 conn(0x7fdde4067e30 msgr2=0x7fdde4069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 shutdown_connections 2023-10-06T11:22:25.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.619+0000 7fdde8d9b700 1 -- 172.21.15.7:0/4217048486 wait complete. 2023-10-06T11:22:25.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.735+0000 7fa5f7c7d700 1 Processor -- start 2023-10-06T11:22:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 -- start start 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa5f0158c40 0x7fa5f0159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5f006ef50 0x7fa5f00683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f0069440 con 0x7fa5f00c7c80 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f0069160 con 0x7fa5f006ef50 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f7c7d700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00c71a0 con 0x7fa5f0158c40 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5f006ef50 0x7fa5f00683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5f006ef50 0x7fa5f00683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40094/0 (socket says 172.21.15.7:40094) 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 -- 172.21.15.7:0/3152824500 learned_addr learned my addr 172.21.15.7:0/3152824500 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f5a19700 1 --2- 172.21.15.7:0/3152824500 >> v2:172.21.15.50:3300/0 conn(0x7fa5f0158c40 0x7fa5f0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 -- 172.21.15.7:0/3152824500 >> v2:172.21.15.50:3300/0 conn(0x7fa5f0158c40 msgr2=0x7fa5f0159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 --2- 172.21.15.7:0/3152824500 >> v2:172.21.15.50:3300/0 conn(0x7fa5f0158c40 0x7fa5f0159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 -- 172.21.15.7:0/3152824500 >> v1:172.21.15.50:6789/0 conn(0x7fa5f00c7c80 legacy=0x7fa5f006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 -- 172.21.15.7:0/3152824500 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00c7480 con 0x7fa5f006ef50 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f621a700 1 --2- 172.21.15.7:0/3152824500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f006ef50 0x7fa5f00683f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0067370 tx=0x7fa5e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8699eead2da3da9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f4a17700 1 -- 172.21.15.7:0/3152824500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800e070 con 0x7fa5f006ef50 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f4a17700 1 -- 172.21.15.7:0/3152824500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e800ba50 con 0x7fa5f006ef50 2023-10-06T11:22:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.739+0000 7fa5f4a17700 1 -- 172.21.15.7:0/3152824500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80092e0 con 0x7fa5f006ef50 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/3152824500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f006ef50 msgr2=0x7fa5f00683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/3152824500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f006ef50 0x7fa5f00683f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0067370 tx=0x7fa5e8009840 comp rx=0 tx=0).stop 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/3152824500 shutdown_connections 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/3152824500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f006ef50 0x7fa5f00683f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/3152824500 >> v2:172.21.15.50:3300/0 conn(0x7fa5f0158c40 0x7fa5f0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/3152824500 >> 172.21.15.7:0/3152824500 conn(0x7fa5f0066840 msgr2=0x7fa5f0066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/3152824500 shutdown_connections 2023-10-06T11:22:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/3152824500 wait complete. 2023-10-06T11:22:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 Processor -- start 2023-10-06T11:22:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.743+0000 7fa5f7c7d700 1 -- start start 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5f006ef50 0x7fa5f0148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5f00c7c80 0x7fa5f00c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0148540 con 0x7fa5f0158c40 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00b0240 con 0x7fa5f00c7c80 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00c5e40 con 0x7fa5f006ef50 2023-10-06T11:22:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40106/0 (socket says 172.21.15.7:40106) 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 -- 172.21.15.7:0/1889954351 learned_addr learned my addr 172.21.15.7:0/1889954351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f5a19700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5f006ef50 0x7fa5f0148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f5218700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5f00c7c80 0x7fa5f00c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 -- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5f006ef50 msgr2=0x7fa5f0148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5f006ef50 0x7fa5f0148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 -- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5f00c7c80 msgr2=0x7fa5f00c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5f00c7c80 0x7fa5f00c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 -- 172.21.15.7:0/1889954351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009ae0 con 0x7fa5f0158c40 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f621a700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8012000 tx=0x7fa5e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800be60 con 0x7fa5f0158c40 2023-10-06T11:22:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e8008d20 con 0x7fa5f0158c40 2023-10-06T11:22:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f00af910 con 0x7fa5f0158c40 2023-10-06T11:22:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80266b0 con 0x7fa5f0158c40 2023-10-06T11:22:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f00afed0 con 0x7fa5f0158c40 2023-10-06T11:22:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.747+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f00b03f0 con 0x7fa5f0158c40 2023-10-06T11:22:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.751+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa5e8003040 con 0x7fa5f0158c40 2023-10-06T11:22:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.751+0000 7fa5e5ffb700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 0x7fa5d4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.751+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa5e807e460 con 0x7fa5f0158c40 2023-10-06T11:22:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.751+0000 7fa5f5a19700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 0x7fa5d4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.751+0000 7fa5f5a19700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 0x7fa5d4062a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006fd0 tx=0x7fa5ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.759+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa5f00b03f0 con 0x7fa5f0158c40 2023-10-06T11:22:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.923+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa5f00b03b0 con 0x7fa5f0158c40 2023-10-06T11:22:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.923+0000 7fa5e5ffb700 1 -- 172.21.15.7:0/1889954351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa5e804e600 con 0x7fa5f0158c40 2023-10-06T11:22:25.927 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.923+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 msgr2=0x7fa5d4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.923+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 0x7fa5d4062a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006fd0 tx=0x7fa5ec008040 comp rx=0 tx=0).stop 2023-10-06T11:22:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 msgr2=0x7fa5f00c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8012000 tx=0x7fa5e8018000 comp rx=0 tx=0).stop 2023-10-06T11:22:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 shutdown_connections 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa5d4060580 0x7fa5d4062a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5f0158c40 0x7fa5f00c40b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5f00c7c80 0x7fa5f00c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 --2- 172.21.15.7:0/1889954351 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5f006ef50 0x7fa5f0148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 >> 172.21.15.7:0/1889954351 conn(0x7fa5f0066840 msgr2=0x7fa5f00caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 shutdown_connections 2023-10-06T11:22:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:25.927+0000 7fa5f7c7d700 1 -- 172.21.15.7:0/1889954351 wait complete. 2023-10-06T11:22:25.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:26.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 Processor -- start 2023-10-06T11:22:26.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 -- start start 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f91b0158fe0 0x7f91b01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 0x7f91b015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b015a5f0 con 0x7f91b006aa30 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b01599e0 con 0x7f91b015d560 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.043+0000 7f91b76ac700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b0159cc0 con 0x7f91b0158fe0 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b4c47700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91b006aa30 0x7f91b015ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35088/0 (socket says 172.21.15.7:35088) 2023-10-06T11:22:26.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b4c47700 1 -- 172.21.15.7:0/780072366 learned_addr learned my addr 172.21.15.7:0/780072366 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:26.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5c49700 1 --2- 172.21.15.7:0/780072366 >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 0x7f91b015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 --2- 172.21.15.7:0/780072366 >> v2:172.21.15.50:3300/0 conn(0x7f91b0158fe0 0x7f91b01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91a7fff700 1 -- 172.21.15.7:0/780072366 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262259459 0 0) 0x7f91b015a5f0 con 0x7f91b006aa30 2023-10-06T11:22:26.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91a7fff700 1 -- 172.21.15.7:0/780072366 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f919c003570 con 0x7f91b006aa30 2023-10-06T11:22:26.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 -- 172.21.15.7:0/780072366 >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 msgr2=0x7f91b015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:26.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 --2- 172.21.15.7:0/780072366 >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 0x7f91b015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 -- 172.21.15.7:0/780072366 >> v1:172.21.15.7:6789/0 conn(0x7f91b006aa30 legacy=0x7f91b015ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 -- 172.21.15.7:0/780072366 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0159fa0 con 0x7f91b0158fe0 2023-10-06T11:22:26.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5c49700 1 --2- 172.21.15.7:0/780072366 >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 0x7f91b015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:26.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b5448700 1 --2- 172.21.15.7:0/780072366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b0158fe0 0x7f91b01593b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91b006d680 tx=0x7f91ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf90b65e6d4d89b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91a7fff700 1 -- 172.21.15.7:0/780072366 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011630 con 0x7f91b0158fe0 2023-10-06T11:22:26.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91a7fff700 1 -- 172.21.15.7:0/780072366 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac0117b0 con 0x7f91b0158fe0 2023-10-06T11:22:26.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91a7fff700 1 -- 172.21.15.7:0/780072366 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011b30 con 0x7f91b0158fe0 2023-10-06T11:22:26.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 -- 172.21.15.7:0/780072366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b0158fe0 msgr2=0x7f91b01593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 --2- 172.21.15.7:0/780072366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b0158fe0 0x7f91b01593b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91b006d680 tx=0x7f91ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 -- 172.21.15.7:0/780072366 shutdown_connections 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 --2- 172.21.15.7:0/780072366 >> v2:172.21.15.7:3300/0 conn(0x7f91b015d560 0x7f91b015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 --2- 172.21.15.7:0/780072366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b0158fe0 0x7f91b01593b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 -- 172.21.15.7:0/780072366 >> 172.21.15.7:0/780072366 conn(0x7f91b0067e30 msgr2=0x7f91b0068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 -- 172.21.15.7:0/780072366 shutdown_connections 2023-10-06T11:22:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.047+0000 7f91b76ac700 1 -- 172.21.15.7:0/780072366 wait complete. 2023-10-06T11:22:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 Processor -- start 2023-10-06T11:22:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 -- start start 2023-10-06T11:22:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b006aa30 0x7f91b01441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 0x7f91b00aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00abf40 con 0x7f91b0158fe0 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00acd80 con 0x7f91b006aa30 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b76ac700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00ad210 con 0x7f91b015d560 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b4c47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b4c47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40124/0 (socket says 172.21.15.7:40124) 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b4c47700 1 -- 172.21.15.7:0/1961167869 learned_addr learned my addr 172.21.15.7:0/1961167869 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b5448700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b006aa30 0x7f91b01441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.051+0000 7f91b5c49700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 0x7f91b00aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 -- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 msgr2=0x7f91b00aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 0x7f91b00aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 -- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b006aa30 msgr2=0x7f91b01441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b006aa30 0x7f91b01441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 -- 172.21.15.7:0/1961167869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009a20 con 0x7f91b0158fe0 2023-10-06T11:22:26.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b5c49700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 0x7f91b00aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b4c47700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91a00011e0 tx=0x7f91a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0005b30 con 0x7f91b0158fe0 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0005c90 con 0x7f91b0158fe0 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac009420 con 0x7f91b0158fe0 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0007320 con 0x7f91b0158fe0 2023-10-06T11:22:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b00ac450 con 0x7f91b0158fe0 2023-10-06T11:22:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b004be00 con 0x7f91b0158fe0 2023-10-06T11:22:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.055+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f91a0007690 con 0x7f91b0158fe0 2023-10-06T11:22:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.059+0000 7f91a57fa700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 0x7f9194062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.059+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f91a007c0b0 con 0x7f91b0158fe0 2023-10-06T11:22:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.059+0000 7f91b5448700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 0x7f9194062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.059+0000 7f91b5448700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 0x7f9194062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91b006d680 tx=0x7f91ac017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.063+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f91a004c250 con 0x7f91b0158fe0 2023-10-06T11:22:26.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.263+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91b015a280 con 0x7f91b0158fe0 2023-10-06T11:22:26.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.263+0000 7f91a57fa700 1 -- 172.21.15.7:0/1961167869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f91a004f870 con 0x7f91b0158fe0 2023-10-06T11:22:26.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:26.270 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 msgr2=0x7f9194062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 0x7f9194062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91b006d680 tx=0x7f91ac017000 comp rx=0 tx=0).stop 2023-10-06T11:22:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 msgr2=0x7f91b00c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91a00011e0 tx=0x7f91a0007df0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 shutdown_connections 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9194060550 0x7f9194062a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91b015d560 0x7f91b00aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0158fe0 0x7f91b00c8620 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 --2- 172.21.15.7:0/1961167869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91b006aa30 0x7f91b01441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 >> 172.21.15.7:0/1961167869 conn(0x7f91b0067e30 msgr2=0x7f91b00af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 shutdown_connections 2023-10-06T11:22:26.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.267+0000 7f91b76ac700 1 -- 172.21.15.7:0/1961167869 wait complete. 2023-10-06T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:26.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 Processor -- start 2023-10-06T11:22:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 -- start start 2023-10-06T11:22:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc48155fd0 0x7fbc481563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc48069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc480c7b20 con 0x7fbc480c9d60 2023-10-06T11:22:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc48159f30 con 0x7fbc48155fd0 2023-10-06T11:22:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc47fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48159710 con 0x7fbc480ca1c0 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc46ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc48155fd0 0x7fbc481563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc46ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc48155fd0 0x7fbc481563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40140/0 (socket says 172.21.15.7:40140) 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc46ffd700 1 -- 172.21.15.7:0/974917045 learned_addr learned my addr 172.21.15.7:0/974917045 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.387+0000 7fbc477fe700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc48069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc46ffd700 1 -- 172.21.15.7:0/974917045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 msgr2=0x7fbc48069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc45ffb700 1 -- 172.21.15.7:0/974917045 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260345385 0 0) 0x7fbc480c7b20 con 0x7fbc480c9d60 2023-10-06T11:22:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc46ffd700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc48069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc46ffd700 1 -- 172.21.15.7:0/974917045 >> v1:172.21.15.50:6789/0 conn(0x7fbc480c9d60 legacy=0x7fbc48159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc46ffd700 1 -- 172.21.15.7:0/974917045 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480c7530 con 0x7fbc48155fd0 2023-10-06T11:22:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc477fe700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc48069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc46ffd700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc481563a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc380099b0 tx=0x7fbc38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6260408bb10dfb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc45ffb700 1 -- 172.21.15.7:0/974917045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38009ea0 con 0x7fbc48155fd0 2023-10-06T11:22:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc45ffb700 1 -- 172.21.15.7:0/974917045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800b860 con 0x7fbc48155fd0 2023-10-06T11:22:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc45ffb700 1 -- 172.21.15.7:0/974917045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800bc00 con 0x7fbc48155fd0 2023-10-06T11:22:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 -- 172.21.15.7:0/974917045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 msgr2=0x7fbc481563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc481563a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc380099b0 tx=0x7fbc38009250 comp rx=0 tx=0).stop 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 -- 172.21.15.7:0/974917045 shutdown_connections 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc48069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 --2- 172.21.15.7:0/974917045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc481563a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 -- 172.21.15.7:0/974917045 >> 172.21.15.7:0/974917045 conn(0x7fbc480cc190 msgr2=0x7fbc480cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 -- 172.21.15.7:0/974917045 shutdown_connections 2023-10-06T11:22:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.391+0000 7fbc47fff700 1 -- 172.21.15.7:0/974917045 wait complete. 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 Processor -- start 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- start start 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc480c9d60 0x7fbc48156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc480b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48148c40 con 0x7fbc48155fd0 2023-10-06T11:22:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48148f20 con 0x7fbc480c9d60 2023-10-06T11:22:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48157360 con 0x7fbc480ca1c0 2023-10-06T11:22:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40150/0 (socket says 172.21.15.7:40150) 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 -- 172.21.15.7:0/3412343287 learned_addr learned my addr 172.21.15.7:0/3412343287 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc46ffd700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc480c9d60 0x7fbc48156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc467fc700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc480b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 -- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 msgr2=0x7fbc480b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc480b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 -- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc480c9d60 msgr2=0x7fbc48156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc480c9d60 0x7fbc48156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 -- 172.21.15.7:0/3412343287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40009a20 con 0x7fbc48155fd0 2023-10-06T11:22:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc477fe700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009f90 tx=0x7fbc4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000cbd0 con 0x7fbc48155fd0 2023-10-06T11:22:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4000f040 con 0x7fbc48155fd0 2023-10-06T11:22:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40014480 con 0x7fbc48155fd0 2023-10-06T11:22:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc380094f0 con 0x7fbc48155fd0 2023-10-06T11:22:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.395+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc48157920 con 0x7fbc48155fd0 2023-10-06T11:22:26.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48157b40 con 0x7fbc48155fd0 2023-10-06T11:22:26.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbc400147f0 con 0x7fbc48155fd0 2023-10-06T11:22:26.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc2effd700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 0x7fbc34062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbc4007d950 con 0x7fbc48155fd0 2023-10-06T11:22:26.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc46ffd700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 0x7fbc34062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.399+0000 7fbc46ffd700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 0x7fbc34062a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc380099b0 tx=0x7fbc38017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.407+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbc4004da50 con 0x7fbc48155fd0 2023-10-06T11:22:26.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.571+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc48149060 con 0x7fbc48155fd0 2023-10-06T11:22:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.571+0000 7fbc2effd700 1 -- 172.21.15.7:0/3412343287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7fbc40018090 con 0x7fbc48155fd0 2023-10-06T11:22:26.577 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:26.578 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 msgr2=0x7fbc34062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 0x7fbc34062a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc380099b0 tx=0x7fbc38017000 comp rx=0 tx=0).stop 2023-10-06T11:22:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 msgr2=0x7fbc480c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009f90 tx=0x7fbc4000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 shutdown_connections 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbc34060580 0x7fbc34062a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48155fd0 0x7fbc480c1f00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbc480ca1c0 0x7fbc480b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 --2- 172.21.15.7:0/3412343287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc480c9d60 0x7fbc48156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 >> 172.21.15.7:0/3412343287 conn(0x7fbc480cc190 msgr2=0x7fbc480c7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 shutdown_connections 2023-10-06T11:22:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.575+0000 7fbc47fff700 1 -- 172.21.15.7:0/3412343287 wait complete. 2023-10-06T11:22:26.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:26.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.691+0000 7fce31c8f700 1 Processor -- start 2023-10-06T11:22:26.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.691+0000 7fce31c8f700 1 -- start start 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce2c0ce300 0x7fce2c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c06a020 con 0x7fce2c15d4b0 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0cd090 con 0x7fce2c0ce300 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0cecc0 con 0x7fce2c0cc720 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce30c8d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce2c15d4b0 0x7fce2c06a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35098/0 (socket says 172.21.15.7:35098) 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 --2- 172.21.15.7:0/4023259143 >> v2:172.21.15.7:3300/0 conn(0x7fce2c0ce300 0x7fce2c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce30c8d700 1 -- 172.21.15.7:0/4023259143 learned_addr learned my addr 172.21.15.7:0/4023259143 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:26.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce2bfff700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 -- 172.21.15.7:0/4023259143 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 msgr2=0x7fce2c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 -- 172.21.15.7:0/4023259143 >> v1:172.21.15.7:6789/0 conn(0x7fce2c15d4b0 legacy=0x7fce2c06a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 -- 172.21.15.7:0/4023259143 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c069730 con 0x7fce2c0ce300 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce3148e700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c0ce780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce24009e60 tx=0x7fce24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f003b76e47f459c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce2b7fe700 1 -- 172.21.15.7:0/4023259143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400e070 con 0x7fce2c0ce300 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce2b7fe700 1 -- 172.21.15.7:0/4023259143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400bb60 con 0x7fce2c0ce300 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce2b7fe700 1 -- 172.21.15.7:0/4023259143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24012430 con 0x7fce2c0ce300 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- 172.21.15.7:0/4023259143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 msgr2=0x7fce2c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c0ce780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce24009e60 tx=0x7fce24009780 comp rx=0 tx=0).stop 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- 172.21.15.7:0/4023259143 shutdown_connections 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c0ce780 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 --2- 172.21.15.7:0/4023259143 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- 172.21.15.7:0/4023259143 >> 172.21.15.7:0/4023259143 conn(0x7fce2c067e30 msgr2=0x7fce2c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- 172.21.15.7:0/4023259143 shutdown_connections 2023-10-06T11:22:26.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.695+0000 7fce31c8f700 1 -- 172.21.15.7:0/4023259143 wait complete. 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 Processor -- start 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 -- start start 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce2c15d4b0 0x7fce2c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c155050 con 0x7fce2c0ce300 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c15c5b0 con 0x7fce2c15d4b0 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0b0800 con 0x7fce2c0cc720 2023-10-06T11:22:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40170/0 (socket says 172.21.15.7:40170) 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 -- 172.21.15.7:0/2748759348 learned_addr learned my addr 172.21.15.7:0/2748759348 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce30c8d700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce3148e700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce2c15d4b0 0x7fce2c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 -- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 msgr2=0x7fce2c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 -- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce2c15d4b0 msgr2=0x7fce2c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce2c15d4b0 0x7fce2c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 -- 172.21.15.7:0/2748759348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24009a20 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce30c8d700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce2bfff700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00ec80 tx=0x7fce1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00c9d0 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00cb30 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c0b0ae0 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.699+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c013470 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c0b23c0 con 0x7fce2c0ce300 2023-10-06T11:22:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c0c6b80 con 0x7fce2c0ce300 2023-10-06T11:22:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fce1c013820 con 0x7fce2c0ce300 2023-10-06T11:22:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce28ff9700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 0x7fce18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce30c8d700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 0x7fce18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fce1c07cfe0 con 0x7fce2c0ce300 2023-10-06T11:22:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.703+0000 7fce30c8d700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 0x7fce18062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce20001820 tx=0x7fce20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.711+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce1c04d100 con 0x7fce2c0ce300 2023-10-06T11:22:26.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.903+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2c04b5c0 con 0x7fce2c0ce300 2023-10-06T11:22:26.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.903+0000 7fce28ff9700 1 -- 172.21.15.7:0/2748759348 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7fce1c050720 con 0x7fce2c0ce300 2023-10-06T11:22:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:26.909 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 msgr2=0x7fce18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 0x7fce18062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce20001820 tx=0x7fce20009040 comp rx=0 tx=0).stop 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 msgr2=0x7fce2c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00ec80 tx=0x7fce1c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 shutdown_connections 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fce18060550 0x7fce18062a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce2c15d4b0 0x7fce2c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce2c0ce300 0x7fce2c15bb60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 --2- 172.21.15.7:0/2748759348 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fce2c0cc720 0x7fce2c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 >> 172.21.15.7:0/2748759348 conn(0x7fce2c067e30 msgr2=0x7fce2c0c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 shutdown_connections 2023-10-06T11:22:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:26.907+0000 7fce31c8f700 1 -- 172.21.15.7:0/2748759348 wait complete. 2023-10-06T11:22:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:27.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:21:56.923085+0000', 'modified': '2023-10-06T11:22:16.238076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/932941616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 932941616}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 932941616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:22:27.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:22:27.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:22:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.439+0000 7f13d2515700 1 Processor -- start 2023-10-06T11:22:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.439+0000 7f13d2515700 1 -- start start 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d2515700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 0x7f13cc06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d2515700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13cc06bea0 0x7f13cc0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d2515700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13cc06e6f0 con 0x7f13cc0cb760 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d2515700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc0cbc70 con 0x7f13cc06bea0 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d2515700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0cbf50 con 0x7f13cc06bad0 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13cc06bea0 0x7f13cc0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d1513700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 0x7f13cc06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13cc06bea0 0x7f13cc0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:36354/0 (socket says 172.21.15.7:36354) 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d1513700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 0x7f13cc06eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:55276/0 (socket says 172.21.15.7:55276) 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d1d14700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f13cc0cb760 0x7f13cc155850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:40568/0 (socket says 172.21.15.7:40568) 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 -- 172.21.15.7:0/2312910159 learned_addr learned my addr 172.21.15.7:0/2312910159 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 -- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 msgr2=0x7f13cc06eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 --2- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 0x7f13cc06eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 -- 172.21.15.7:0/2312910159 >> v1:172.21.15.50:6789/0 conn(0x7f13cc0cb760 legacy=0x7f13cc155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 -- 172.21.15.7:0/2312910159 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc06e0a0 con 0x7f13cc06bea0 2023-10-06T11:22:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13d0d12700 1 --2- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc0cb220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0099b0 tx=0x7f13bc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc99d522bc8c5a5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.443+0000 7f13c3fff700 1 -- 172.21.15.7:0/2312910159 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc009ea0 con 0x7f13cc06bea0 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13c3fff700 1 -- 172.21.15.7:0/2312910159 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc00b860 con 0x7f13cc06bea0 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13c3fff700 1 -- 172.21.15.7:0/2312910159 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00bc00 con 0x7f13cc06bea0 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 -- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 msgr2=0x7f13cc0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 --2- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc0cb220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0099b0 tx=0x7f13bc009250 comp rx=0 tx=0).stop 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 -- 172.21.15.7:0/2312910159 shutdown_connections 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 --2- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc0cb220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 --2- 172.21.15.7:0/2312910159 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc06bad0 0x7f13cc06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 -- 172.21.15.7:0/2312910159 >> 172.21.15.7:0/2312910159 conn(0x7f13cc0cc190 msgr2=0x7f13cc0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 -- 172.21.15.7:0/2312910159 shutdown_connections 2023-10-06T11:22:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.447+0000 7f13d2515700 1 -- 172.21.15.7:0/2312910159 wait complete. 2023-10-06T11:22:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 Processor -- start 2023-10-06T11:22:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 -- start start 2023-10-06T11:22:27.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc0cb760 0x7f13cc14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc157bc0 con 0x7f13cc06bad0 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc158a00 con 0x7f13cc06bea0 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc158e90 con 0x7f13cc0cb760 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40176/0 (socket says 172.21.15.7:40176) 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 -- 172.21.15.7:0/4150079692 learned_addr learned my addr 172.21.15.7:0/4150079692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1d14700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc0cb760 0x7f13cc14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d0d12700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 -- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc0cb760 msgr2=0x7f13cc14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc0cb760 0x7f13cc14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 -- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 msgr2=0x7f13cc157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 -- 172.21.15.7:0/4150079692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc0094f0 con 0x7f13cc06bad0 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d1513700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009f10 tx=0x7f13c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13c17fa700 1 -- 172.21.15.7:0/4150079692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c400cb70 con 0x7f13cc06bad0 2023-10-06T11:22:27.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13c17fa700 1 -- 172.21.15.7:0/4150079692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c400f040 con 0x7f13cc06bad0 2023-10-06T11:22:27.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc157df0 con 0x7f13cc06bad0 2023-10-06T11:22:27.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.451+0000 7f13c17fa700 1 -- 172.21.15.7:0/4150079692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4014480 con 0x7f13cc06bad0 2023-10-06T11:22:27.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.455+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc1583b0 con 0x7f13cc06bad0 2023-10-06T11:22:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.455+0000 7f13c17fa700 1 -- 172.21.15.7:0/4150079692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f13c4014830 con 0x7f13cc06bad0 2023-10-06T11:22:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.455+0000 7f13c17fa700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 0x7f13b0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.455+0000 7f13c17fa700 1 -- 172.21.15.7:0/4150079692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f13c407d5c0 con 0x7f13cc06bad0 2023-10-06T11:22:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.455+0000 7f13d0d12700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 0x7f13b0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 0x7f13b0062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0099b0 tx=0x7f13bc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 Processor -- start 2023-10-06T11:22:27.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 -- start start 2023-10-06T11:22:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2868c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x2792f30 con 0x25e7710 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x278eed0 con 0x25e7ae0 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x27550b0 con 0x2632790 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40192/0 (socket says 172.21.15.7:40192) 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 -- 172.21.15.7:0/880761321 learned_addr learned my addr 172.21.15.7:0/880761321 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d1513700 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2868c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d1d14700 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 -- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 msgr2=0x28696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 -- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 msgr2=0x2868c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2868c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 -- 172.21.15.7:0/880761321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26339e0 con 0x25e7710 2023-10-06T11:22:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13d0d12700 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006fd0 tx=0x7f13bc01d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c117332eb3280d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13b67fc700 1 -- 172.21.15.7:0/880761321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc02d070 con 0x25e7710 2023-10-06T11:22:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.459+0000 7f13b67fc700 1 -- 172.21.15.7:0/880761321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc021040 con 0x25e7710 2023-10-06T11:22:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13b67fc700 1 -- 172.21.15.7:0/880761321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc01bb90 con 0x25e7710 2023-10-06T11:22:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- 172.21.15.7:0/880761321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 msgr2=0x2869170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006fd0 tx=0x7f13bc01d050 comp rx=0 tx=0).stop 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- 172.21.15.7:0/880761321 shutdown_connections 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x2869170 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- 172.21.15.7:0/880761321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2868c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- 172.21.15.7:0/880761321 >> 172.21.15.7:0/880761321 conn(0x25e7eb0 msgr2=0x2821060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- 172.21.15.7:0/880761321 shutdown_connections 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- 172.21.15.7:0/880761321 wait complete. 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- start start 2023-10-06T11:22:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 Processor -- start 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x281ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x25e7ae0 0x281f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x2754d50 con 0x2632790 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x272d970 con 0x25e7710 2023-10-06T11:22:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x25be060 con 0x25e7ae0 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d5a20740 10 client.0 _ll_register_callbacks cb 0x2731b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d1d14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d1d14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40206/0 (socket says 172.21.15.7:40206) 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d1d14700 1 -- 172.21.15.7:0/327821055 learned_addr learned my addr 172.21.15.7:0/327821055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d0d12700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x25e7ae0 0x281f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.463+0000 7f13d1513700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x281ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 -- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x25e7ae0 msgr2=0x281f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x25e7ae0 0x281f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 -- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 msgr2=0x281ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x281ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 -- 172.21.15.7:0/327821055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc06e0a0 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1d14700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13c8007850 tx=0x7f13c8007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade2fc096ab0502 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c801a030 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f13c801b070 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x272dcc0 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c8021470 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 10 client.4821 fetch_fsmap learned FSMap version 9 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x275d370 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8025020 con 0x2632790 2023-10-06T11:22:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f13c8015440 con 0x2632790 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 10 client.4821 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 20 client.4821 populate_metadata read hostname 'smithi007' 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28756a0 0x2877a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 4 client.4821 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 --> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27313e0 con 0x28756a0 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1513700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28756a0 0x2877a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 10 client.4821 ms_handle_connect on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d1513700 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28756a0 0x2877a80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa1d4d822b4caa7 server_cookie=18cbe3360034aab2 in_seq=0 out_seq=0 2023-10-06T11:22:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 1 -- 172.21.15.7:0/327821055 <== mds.0 v2:172.21.15.7:6834/932941616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x27313e0 con 0x28756a0 2023-10-06T11:22:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13ab7fe700 10 client.4821 handle_command_reply: tid=0 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 client.4821 shutdown 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 msgr2=0x281f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13c8007850 tx=0x7f13c8007e90 comp rx=0 tx=0).stop 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 shutdown_connections 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28756a0 0x2877a80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2632790 0x281f6e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.467+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x25e7ae0 0x281f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13ab7fe700 0 client.4821 ms_handle_reset on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13d5a20740 1 --2- 172.21.15.7:0/327821055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x281ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 >> 172.21.15.7:0/327821055 conn(0x25e7eb0 msgr2=0x28461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 shutdown_connections 2023-10-06T11:22:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13d5a20740 1 -- 172.21.15.7:0/327821055 wait complete. 2023-10-06T11:22:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.471+0000 7f13d5a20740 20 client.4821 trim_cache size 0 max 16384 2023-10-06T11:22:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 Processor -- start 2023-10-06T11:22:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- start start 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2820d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x281ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x2762040 con 0x25e7710 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b6110 con 0x25e7ae0 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x278ab50 con 0x2632790 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40220/0 (socket says 172.21.15.7:40220) 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 -- 172.21.15.7:0/279141496 learned_addr learned my addr 172.21.15.7:0/279141496 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1d14700 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x281ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d0d12700 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2820d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 -- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 msgr2=0x281ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x281ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 -- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 msgr2=0x2820d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2820d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 -- 172.21.15.7:0/279141496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0afc50 con 0x25e7710 2023-10-06T11:22:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d1513700 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f13c4051310 tx=0x7f13c401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90362da7e9088a08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13b6ffd700 1 -- 172.21.15.7:0/279141496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4051380 con 0x25e7710 2023-10-06T11:22:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13b6ffd700 1 -- 172.21.15.7:0/279141496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c404f4c0 con 0x25e7710 2023-10-06T11:22:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13b6ffd700 1 -- 172.21.15.7:0/279141496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c404f720 con 0x25e7710 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- 172.21.15.7:0/279141496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 msgr2=0x28207e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f13c4051310 tx=0x7f13c401a000 comp rx=0 tx=0).stop 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- 172.21.15.7:0/279141496 shutdown_connections 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x281ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7ae0 0x2820d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 --2- 172.21.15.7:0/279141496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7710 0x28207e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- 172.21.15.7:0/279141496 >> 172.21.15.7:0/279141496 conn(0x25e7eb0 msgr2=0x28200d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- 172.21.15.7:0/279141496 shutdown_connections 2023-10-06T11:22:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.491+0000 7f13d5a20740 1 -- 172.21.15.7:0/279141496 wait complete. 2023-10-06T11:22:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 -- start start 2023-10-06T11:22:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 Processor -- start 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x2820a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 0x28a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x27e5270 con 0x25e7ae0 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x2766d60 con 0x25e7710 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x2667650 con 0x2632790 2023-10-06T11:22:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d5a20740 10 client.0 _ll_register_callbacks cb 0x275a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:22:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.495+0000 7f13d0d12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 0x28a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d0d12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 0x28a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40228/0 (socket says 172.21.15.7:40228) 2023-10-06T11:22:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1513700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x2820a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d0d12700 1 -- 172.21.15.7:0/526255832 learned_addr learned my addr 172.21.15.7:0/526255832 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 -- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 msgr2=0x2820a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x2820a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 -- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 msgr2=0x28a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 0x28a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 -- 172.21.15.7:0/526255832 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc1583b0 con 0x2632790 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1d14700 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28a6e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13c8007200 tx=0x7f13c8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39c9e5ff0535b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.50:3301/0 2023-10-06T11:22:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c802f070 con 0x2632790 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mon.2 v2:172.21.15.50:3301/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f13c8026080 con 0x2632790 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13bc021040 con 0x2632790 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c801a050 con 0x2632790 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 10 client.4631 fetch_fsmap learned FSMap version 9 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x2743d80 con 0x2632790 2023-10-06T11:22:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mon.2 v2:172.21.15.50:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc021040 con 0x2632790 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mon.2 v2:172.21.15.50:3301/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f13c8032020 con 0x2632790 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 10 client.4631 fetch_fsmap finished waiting for FSMap version 9 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 10 client.4631 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 20 client.4631 populate_metadata read hostname 'smithi007' 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28a7370 0x28c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 4 client.4631 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-10-06T11:22:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 --> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2763410 con 0x28a7370 2023-10-06T11:22:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1513700 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28a7370 0x28c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13ab7fe700 10 client.4631 ms_handle_connect on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.499+0000 7f13d1513700 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28a7370 0x28c21d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d04bff531abdca server_cookie=b02e3691134abad2 in_seq=0 out_seq=0 2023-10-06T11:22:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13ab7fe700 1 -- 172.21.15.7:0/526255832 <== mds.0 v2:172.21.15.7:6834/932941616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x2763410 con 0x28a7370 2023-10-06T11:22:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13ab7fe700 10 client.4631 handle_command_reply: tid=0 2023-10-06T11:22:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 client.4631 shutdown 2023-10-06T11:22:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 msgr2=0x28a6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28a6e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13c8007200 tx=0x7f13c8017000 comp rx=0 tx=0).stop 2023-10-06T11:22:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 shutdown_connections 2023-10-06T11:22:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:6834/932941616,v1:172.21.15.7:6835/932941616] conn(0x28a7370 0x28c21d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x2632790 0x28a6e30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13ab7fe700 0 client.4631 ms_handle_reset on v2:172.21.15.7:6834/932941616 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x25e7ae0 0x28a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 --2- 172.21.15.7:0/526255832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x25e7710 0x2820a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 >> 172.21.15.7:0/526255832 conn(0x25e7eb0 msgr2=0x281ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 shutdown_connections 2023-10-06T11:22:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 1 -- 172.21.15.7:0/526255832 wait complete. 2023-10-06T11:22:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d5a20740 20 client.4631 trim_cache size 0 max 16384 2023-10-06T11:22:27.513 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:27.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "88789558-6143-4e37-9ef9-c1260e585533", 2023-10-06T11:22:27.513 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4287, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 29.933926811999999, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 9, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 25, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 25, 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 35.249202285000003 2023-10-06T11:22:27.514 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 msgr2=0x7f13b0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 0x7f13b0062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0099b0 tx=0x7f13bc017000 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 msgr2=0x7f13cc0c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009f10 tx=0x7f13c400b040 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 shutdown_connections 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13b0060550 0x7f13b0062a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13cc0cb760 0x7f13cc14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13cc06bea0 0x7f13cc157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 --2- 172.21.15.7:0/4150079692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc06bad0 0x7f13cc0c5b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 >> 172.21.15.7:0/4150079692 conn(0x7f13cc0cc190 msgr2=0x7f13cc0c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 shutdown_connections 2023-10-06T11:22:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.503+0000 7f13d2515700 1 -- 172.21.15.7:0/4150079692 wait complete. 2023-10-06T11:22:27.525 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T11:22:27.536 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T11:22:27.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:27.537 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-06T11:22:27.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:27.545 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2023-10-06T11:22:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2023-10-06T11:22:27.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:22:27.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:27.567 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:22:27.575 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-06T11:22:27.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:27.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T11:22:27.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:27.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-10-06T11:22:27.775 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T11:22:27.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:22:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.875+0000 7f4b83f35700 1 Processor -- start 2023-10-06T11:22:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.875+0000 7f4b83f35700 1 -- start start 2023-10-06T11:22:27.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c06c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b7c0ae540 0x7f4b7c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4b7c0c91d0 0x7f4b7c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7c0afeb0 con 0x7f4b7c0ae540 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7c0af2a0 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0af580 con 0x7f4b7c155200 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b7c0ae540 0x7f4b7c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b7c0ae540 0x7f4b7c06e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40242/0 (socket says 172.21.15.7:40242) 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 -- 172.21.15.7:0/634088720 learned_addr learned my addr 172.21.15.7:0/634088720 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b81cd1700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c06c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b824d2700 1 --2- 172.21.15.7:0/634088720 >> v2:172.21.15.50:3300/0 conn(0x7f4b7c0c91d0 0x7f4b7c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 -- 172.21.15.7:0/634088720 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 msgr2=0x7f4b7c06c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c06c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 -- 172.21.15.7:0/634088720 >> v2:172.21.15.50:3300/0 conn(0x7f4b7c0c91d0 msgr2=0x7f4b7c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 --2- 172.21.15.7:0/634088720 >> v2:172.21.15.50:3300/0 conn(0x7f4b7c0c91d0 0x7f4b7c06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 -- 172.21.15.7:0/634088720 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0af860 con 0x7f4b7c0ae540 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b814d0700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c06e200 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c06b650 tx=0x7f4b78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b8489efe7219c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b80ccf700 1 -- 172.21.15.7:0/634088720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800b6d0 con 0x7f4b7c0ae540 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b80ccf700 1 -- 172.21.15.7:0/634088720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7800b830 con 0x7f4b7c0ae540 2023-10-06T11:22:27.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b80ccf700 1 -- 172.21.15.7:0/634088720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800bbf0 con 0x7f4b7c0ae540 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- 172.21.15.7:0/634088720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0ae540 msgr2=0x7f4b7c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c06e200 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c06b650 tx=0x7f4b78009250 comp rx=0 tx=0).stop 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- 172.21.15.7:0/634088720 shutdown_connections 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- 172.21.15.7:0/634088720 >> v2:172.21.15.50:3300/0 conn(0x7f4b7c0c91d0 0x7f4b7c06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c06e200 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 --2- 172.21.15.7:0/634088720 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c06c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- 172.21.15.7:0/634088720 >> 172.21.15.7:0/634088720 conn(0x7f4b7c0ccdc0 msgr2=0x7f4b7c0add50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:27.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- 172.21.15.7:0/634088720 shutdown_connections 2023-10-06T11:22:27.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.879+0000 7f4b83f35700 1 -- 172.21.15.7:0/634088720 wait complete. 2023-10-06T11:22:27.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 Processor -- start 2023-10-06T11:22:27.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 -- start start 2023-10-06T11:22:27.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1516c0 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c151090 con 0x7f4b7c0ae540 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b83f35700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1657b0 con 0x7f4b7c155200 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40256/0 (socket says 172.21.15.7:40256) 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 -- 172.21.15.7:0/138782358 learned_addr learned my addr 172.21.15.7:0/138782358 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b81cd1700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b824d2700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 -- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 msgr2=0x7f4b7c164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c164c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 -- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b7c0ae540 msgr2=0x7f4b7c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b814d0700 1 -- 172.21.15.7:0/138782358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780094f0 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.883+0000 7f4b824d2700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b814d0700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b780094c0 tx=0x7f4b78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7801e070 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78000c40 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c165a90 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800bbf0 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c0c3cd0 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c04be00 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4b78022070 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 0x7f4b60062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4b78014070 con 0x7f4b7c0c91d0 2023-10-06T11:22:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.887+0000 7f4b81cd1700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 0x7f4b60062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.891+0000 7f4b81cd1700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 0x7f4b60062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c06c4f0 tx=0x7f4b6c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:27.895+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b7804fe30 con 0x7f4b7c0c91d0 2023-10-06T11:22:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.095+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b7c0afb40 con 0x7f4b7c0c91d0 2023-10-06T11:22:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.095+0000 7f4b71ffb700 1 -- 172.21.15.7:0/138782358 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4b78029020 con 0x7f4b7c0c91d0 2023-10-06T11:22:28.101 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 msgr2=0x7f4b60062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 0x7f4b60062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c06c4f0 tx=0x7f4b6c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 msgr2=0x7f4b7c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b780094c0 tx=0x7f4b78019000 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 shutdown_connections 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b60060550 0x7f4b60062a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b7c155200 0x7f4b7c164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7c0c91d0 0x7f4b7c0c3790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 --2- 172.21.15.7:0/138782358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b7c0ae540 0x7f4b7c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 >> 172.21.15.7:0/138782358 conn(0x7f4b7c0ccdc0 msgr2=0x7f4b7c0c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 shutdown_connections 2023-10-06T11:22:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:28.099+0000 7f4b83f35700 1 -- 172.21.15.7:0/138782358 wait complete. 2023-10-06T11:22:28.115 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:22:28.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:22:28.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-06T11:22:28.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:22:28.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:28.116 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:22:28.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:22:28.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:22:28.116 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:22:28.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.116 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-06T11:22:28.123 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:28.124 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42554sec preferred_lft 42554sec 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:c9:9f:83:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:22:28.125 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:28.126 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:22:28.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:22:28.126 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:28.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.166 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:22:28.224 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-06T11:22:28.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.226 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:22:28.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:22:28.236 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:28.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.276 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:22:28.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.283 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-10-06T11:22:28.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.290 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:28.290 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:28.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:22:28.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T11:22:28.290 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:28.361 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T11:22:28.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T11:22:28.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T11:22:28.363 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:28.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-06T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:22:28.421 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:28.445 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T11:22:28.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:28.446 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:28.512 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:22:28.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:28.512 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:28.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:28.533 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:28.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:28.550 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:28.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:28.551 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:28.557 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:22:28.557 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T11:22:28.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:28.615 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:28.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:28.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:28.627 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:28.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-06T11:22:28.679+0000 7f1ed3c88380 -1 init, newargv = 0x55a6b4420c20 newargc=15 2023-10-06T11:22:28.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[40990]: starting ceph client 2023-10-06T11:22:28.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[40990]: starting fuse 2023-10-06T11:22:29.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:29.752 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:29.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:29.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:29.769 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:29.774 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:22:29.774 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:29.777 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:22:29.778 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-06T11:22:29.779 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-06T11:22:29.860 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.40990.asok 2023-10-06T11:22:30.046 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40990.asok 2023-10-06T11:22:30.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:30.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40990.asok status 2023-10-06T11:22:30.188 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-06T11:22:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:22:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "40990", 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4839, 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-06T11:22:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4839 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2802997681 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2802997681 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4839 192.168.0.1:0/2802997681", 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/2802997681", 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 9, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 25, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-06T11:22:30.191 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:30.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:22:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 Processor -- start 2023-10-06T11:22:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 -- start start 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98d00b1390 0x7f98d0163e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00b07f0 con 0x7f98d00af700 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00afbe0 con 0x7f98d0158df0 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98d4ea0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d00afec0 con 0x7f98d00b1390 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cdd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f98d00af700 0x7f98d00b0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35114/0 (socket says 172.21.15.7:35114) 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cdd9b700 1 -- 172.21.15.7:0/1236342605 learned_addr learned my addr 172.21.15.7:0/1236342605 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:30.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98ced9d700 1 --2- 172.21.15.7:0/1236342605 >> v2:172.21.15.7:3300/0 conn(0x7f98d00b1390 0x7f98d0163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:30.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799014274 0 0) 0x7f98d00b07f0 con 0x7f98d00af700 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98bc003570 con 0x7f98d00af700 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12876379 0 0) 0x7f98bc003570 con 0x7f98d00af700 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98ced9d700 1 -- 172.21.15.7:0/1236342605 >> v1:172.21.15.50:6789/0 conn(0x7f98d0158df0 legacy=0x7f98d00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98ced9d700 1 -- 172.21.15.7:0/1236342605 >> v1:172.21.15.7:6789/0 conn(0x7f98d00af700 legacy=0x7f98d00b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98ced9d700 1 -- 172.21.15.7:0/1236342605 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00b01a0 con 0x7f98d00b1390 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98ced9d700 1 --2- 172.21.15.7:0/1236342605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0163e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098e0 tx=0x7f98c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a34798b0d5ef85d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c80017f0 con 0x7f98d00b1390 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0011420 con 0x7f98d00b1390 2023-10-06T11:22:30.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.299+0000 7f98cd59a700 1 -- 172.21.15.7:0/1236342605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00117c0 con 0x7f98d00b1390 2023-10-06T11:22:30.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1236342605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 msgr2=0x7f98d0163e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1236342605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0163e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098e0 tx=0x7f98c00091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1236342605 shutdown_connections 2023-10-06T11:22:30.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1236342605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0163e90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1236342605 >> 172.21.15.7:0/1236342605 conn(0x7f98d00688c0 msgr2=0x7f98d00cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1236342605 shutdown_connections 2023-10-06T11:22:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1236342605 wait complete. 2023-10-06T11:22:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 Processor -- start 2023-10-06T11:22:30.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.303+0000 7f98d4ea0700 1 -- start start 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d00af700 0x7f98d00ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f98d0158df0 0x7f98d006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00ae4b0 con 0x7f98d00b1390 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00adae0 con 0x7f98d00af700 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00a9500 con 0x7f98d0158df0 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40280/0 (socket says 172.21.15.7:40280) 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 -- 172.21.15.7:0/1848639320 learned_addr learned my addr 172.21.15.7:0/1848639320 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98ced9d700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f98d0158df0 0x7f98d006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98ce59c700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d00af700 0x7f98d00ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 -- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f98d0158df0 msgr2=0x7f98d006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f98d0158df0 0x7f98d006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 -- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d00af700 msgr2=0x7f98d00ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d00af700 0x7f98d00ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 -- 172.21.15.7:0/1848639320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009420 con 0x7f98d00b1390 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98cdd9b700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98c800cf80 tx=0x7f98c8001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800abf0 con 0x7f98d00b1390 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c800ad70 con 0x7f98d00b1390 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d00acb20 con 0x7f98d00b1390 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8007340 con 0x7f98d00b1390 2023-10-06T11:22:30.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d00ac6e0 con 0x7f98d00b1390 2023-10-06T11:22:30.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.307+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d00ad2a0 con 0x7f98d00b1390 2023-10-06T11:22:30.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.311+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f98c80076b0 con 0x7f98d00b1390 2023-10-06T11:22:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.311+0000 7f98c67fc700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 0x7f98b0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.311+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f98c807d2a0 con 0x7f98d00b1390 2023-10-06T11:22:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.311+0000 7f98ce59c700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 0x7f98b0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.311+0000 7f98ce59c700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 0x7f98b0062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98b80039e0 tx=0x7f98b8007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:30.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.319+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98c804d4c0 con 0x7f98d00b1390 2023-10-06T11:22:30.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.515+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98d004be00 con 0x7f98d00b1390 2023-10-06T11:22:30.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.515+0000 7f98c67fc700 1 -- 172.21.15.7:0/1848639320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f98c8050ae0 con 0x7f98d00b1390 2023-10-06T11:22:30.521 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 msgr2=0x7f98b0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 0x7f98b0062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98b80039e0 tx=0x7f98b8007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 msgr2=0x7f98d0153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98c800cf80 tx=0x7f98c8001d90 comp rx=0 tx=0).stop 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 shutdown_connections 2023-10-06T11:22:30.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f98b0060550 0x7f98b0062a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f98d0158df0 0x7f98d006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d00b1390 0x7f98d0153950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 --2- 172.21.15.7:0/1848639320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d00af700 0x7f98d00ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 >> 172.21.15.7:0/1848639320 conn(0x7f98d00688c0 msgr2=0x7f98d01548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 shutdown_connections 2023-10-06T11:22:30.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:30.519+0000 7f98d4ea0700 1 -- 172.21.15.7:0/1848639320 wait complete. 2023-10-06T11:22:30.540 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:22:30.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:22:30.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:22:30.541 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:22:30.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.541 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-06T11:22:30.549 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:22:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42551sec preferred_lft 42551sec 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:c9:9f:83:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:22:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ee:6e:ff:5d:f0:75 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::68f6:b5ff:fe13:5c60/64 scope link 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ee:6e:ff:5d:f0:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec6e:ffff:fe5d:f075/64 scope link 2023-10-06T11:22:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.553 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:22:30.608 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-06T11:22:30.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.610 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-06T11:22:30.619 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-06T11:22:30.619 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:22:30.620 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:22:30.620 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:22:30.620 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:22:30.620 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:22:30.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.620 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:30.620 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:30.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:22:30.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:22:30.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:22:30.621 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:30.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.658 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:22:30.665 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:22:30.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.666 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-10-06T11:22:30.716 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T11:22:30.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.717 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:30.717 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:30.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:22:30.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T11:22:30.717 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:22:30.759 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 22:24:1f:6f:da:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2024:1fff:fe6f:dae5/64 scope link 2023-10-06T11:22:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:22:30.822 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 22:24:1f:6f:da:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2024:1fff:fe6f:dae5/64 scope link 2023-10-06T11:22:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:30.825 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T11:22:30.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T11:22:30.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T11:22:30.826 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:30.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:30.881 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:22:30.881 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:22:30.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2023-10-06T11:22:30.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:22:30.881 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:22:30.908 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T11:22:30.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:30.908 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:30.957 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:22:30.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:30.957 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:30.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:30.980 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:30.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:31.038 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:31.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:31.039 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:31.046 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:22:31.046 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-06T11:22:31.046 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T11:22:31.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:31.062 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:31.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:31.122 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:31.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:31.123 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:31.130 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:22:31.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:2023-10-06T11:22:31.127+0000 7f2e1493e380 -1 init, newargv = 0x560990200a20 newargc=15 2023-10-06T11:22:31.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[41128]: starting ceph client 2023-10-06T11:22:31.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[41128]: starting fuse 2023-10-06T11:22:32.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:22:32.251 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:32.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:32.267 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:32.268 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:32.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:32.268 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:22:32.275 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:22:32.276 INFO:teuthology.orchestra.run.smithi007.stdout:53 2023-10-06T11:22:32.276 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:32.276 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:22:32.277 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.2")) 2023-10-06T11:22:32.278 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-06T11:22:32.403 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.2.41128.asok 2023-10-06T11:22:32.588 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.41128.asok 2023-10-06T11:22:32.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:32.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.41128.asok status 2023-10-06T11:22:32.683 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "2", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "41128", 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-06T11:22:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4854, 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4854 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:22:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1282077277 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1282077277 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4854 192.168.0.2:0/1282077277", 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.2:0/1282077277", 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-06T11:22:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 9, 2023-10-06T11:22:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 25, 2023-10-06T11:22:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:22:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-06T11:22:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-06T11:22:32.687 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:32.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:22:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 Processor -- start 2023-10-06T11:22:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- start start 2023-10-06T11:22:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb5340cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5340c07c0 0x7fb5340ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53415b5a0 con 0x7fb5341567e0 2023-10-06T11:22:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5340caea0 con 0x7fb5340c07c0 2023-10-06T11:22:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53415a990 con 0x7fb5340cbe20 2023-10-06T11:22:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5340c07c0 0x7fb5340ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5340c07c0 0x7fb5340ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44470/0 (socket says 172.21.15.7:44470) 2023-10-06T11:22:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 -- 172.21.15.7:0/4290622908 learned_addr learned my addr 172.21.15.7:0/4290622908 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb53259c700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb5340cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 -- 172.21.15.7:0/4290622908 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 msgr2=0x7fb5340cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb5340cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 -- 172.21.15.7:0/4290622908 >> v1:172.21.15.50:6789/0 conn(0x7fb5341567e0 legacy=0x7fb534156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 -- 172.21.15.7:0/4290622908 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53415ac70 con 0x7fb5340c07c0 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb531d9b700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb5340ca960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5280099b0 tx=0x7fb528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff7c635280f2c02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb53159a700 1 -- 172.21.15.7:0/4290622908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528009ea0 con 0x7fb5340c07c0 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb53159a700 1 -- 172.21.15.7:0/4290622908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800b860 con 0x7fb5340c07c0 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb53159a700 1 -- 172.21.15.7:0/4290622908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800bc00 con 0x7fb5340c07c0 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- 172.21.15.7:0/4290622908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 msgr2=0x7fb5340ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb5340ca960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5280099b0 tx=0x7fb528009250 comp rx=0 tx=0).stop 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- 172.21.15.7:0/4290622908 shutdown_connections 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb5340ca960 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 --2- 172.21.15.7:0/4290622908 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb5340cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- 172.21.15.7:0/4290622908 >> 172.21.15.7:0/4290622908 conn(0x7fb534066840 msgr2=0x7fb534066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- 172.21.15.7:0/4290622908 shutdown_connections 2023-10-06T11:22:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.795+0000 7fb538ca7700 1 -- 172.21.15.7:0/4290622908 wait complete. 2023-10-06T11:22:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 Processor -- start 2023-10-06T11:22:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 -- start start 2023-10-06T11:22:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb534155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5341567e0 0x7fb53414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53414c130 con 0x7fb5340c07c0 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb534156340 con 0x7fb5341567e0 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb538ca7700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb534155970 con 0x7fb5340cbe20 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44476/0 (socket says 172.21.15.7:44476) 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 -- 172.21.15.7:0/3183967880 learned_addr learned my addr 172.21.15.7:0/3183967880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb531d9b700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb534155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb532d9d700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5341567e0 0x7fb53414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 -- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 msgr2=0x7fb534155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb534155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 -- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5341567e0 msgr2=0x7fb53414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5341567e0 0x7fb53414d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 -- 172.21.15.7:0/3183967880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280094f0 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb53259c700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb52400ec80 tx=0x7fb5240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400c850 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.799+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400c9b0 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.803+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb534155040 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.803+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400cd50 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.803+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb534155600 con 0x7fb5340c07c0 2023-10-06T11:22:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.803+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53404be00 con 0x7fb5340c07c0 2023-10-06T11:22:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.807+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb524013680 con 0x7fb5340c07c0 2023-10-06T11:22:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.807+0000 7fb5227fc700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.807+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb52407d690 con 0x7fb5340c07c0 2023-10-06T11:22:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.807+0000 7fb531d9b700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.807+0000 7fb531d9b700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 0x7fb514062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb5280099b0 tx=0x7fb528017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:32.811+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb52404d830 con 0x7fb5340c07c0 2023-10-06T11:22:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb53415af50 con 0x7fb5340c07c0 2023-10-06T11:22:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb5227fc700 1 -- 172.21.15.7:0/3183967880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb524050e50 con 0x7fb5340c07c0 2023-10-06T11:22:33.015 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T11:22:33.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 msgr2=0x7fb514062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:33.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 0x7fb514062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb5280099b0 tx=0x7fb528017000 comp rx=0 tx=0).stop 2023-10-06T11:22:33.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 msgr2=0x7fb53414ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:33.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb52400ec80 tx=0x7fb5240091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:33.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 shutdown_connections 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5341567e0 0x7fb53414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb5340cbe20 0x7fb534155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 --2- 172.21.15.7:0/3183967880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5340c07c0 0x7fb53414ad40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 >> 172.21.15.7:0/3183967880 conn(0x7fb534066840 msgr2=0x7fb534157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 shutdown_connections 2023-10-06T11:22:33.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:33.011+0000 7fb538ca7700 1 -- 172.21.15.7:0/3183967880 wait complete. 2023-10-06T11:22:33.029 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:22:33.030 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:22:33.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.031 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:33.039 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42558sec preferred_lft 42558sec 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:33.040 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:3c:4d:3d:a4 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:33.041 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:22:33.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.041 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:22:33.042 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:33.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.130 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:22:33.141 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-06T11:22:33.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.143 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:22:33.165 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:22:33.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:22:33.166 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:33.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.252 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-06T11:22:33.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.258 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2023-10-06T11:22:33.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.265 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:33.265 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:33.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:22:33.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T11:22:33.265 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:33.336 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T11:22:33.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:22:33.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T11:22:33.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T11:22:33.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T11:22:33.337 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:33.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:33.404 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:33.404 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:33.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2023-10-06T11:22:33.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:22:33.404 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:33.431 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T11:22:33.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:33.432 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:33.461 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:22:33.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:33.462 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:33.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:33.527 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:33.527 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:33.537 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:33.538 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:33.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:33.539 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:33.588 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:22:33.589 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T11:22:33.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:33.611 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:33.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:33.625 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:33.626 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:33.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:33.626 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:33.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi050.stderr:2023-10-06T11:22:33.675+0000 7f61e2c7f380 -1 init, newargv = 0x55dacdfe8ca0 newargc=15 2023-10-06T11:22:33.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi050.stderr:ceph-fuse[38985]: starting ceph client 2023-10-06T11:22:33.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi050.stderr:ceph-fuse[38985]: starting fuse 2023-10-06T11:22:34.777 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:34.795 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:34.795 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:34.811 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:34.811 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:34.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:34.812 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:34.864 INFO:teuthology.orchestra.run.smithi050.stdout:52 2023-10-06T11:22:34.865 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:34.865 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:22:34.866 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.1")) 2023-10-06T11:22:34.867 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-06T11:22:34.949 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.1.38985.asok 2023-10-06T11:22:35.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:35.092 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:35.133 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38985.asok 2023-10-06T11:22:35.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:35.134 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38985.asok status 2023-10-06T11:22:35.285 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "1", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "38985", 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-06T11:22:35.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4658, 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4658 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:35.287 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2872096571 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2872096571 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4658 192.168.0.1:0/2872096571", 2023-10-06T11:22:35.288 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/2872096571", 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 9, 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 25, 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-06T11:22:35.289 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-06T11:22:35.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:22:35.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 Processor -- start 2023-10-06T11:22:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 -- start start 2023-10-06T11:22:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fab440b5560 0x7fab440a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab440af6d0 0x7fab440a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab440a94c0 con 0x7fab440d0b20 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440a88b0 con 0x7fab440af6d0 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab4b01d700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440a8b90 con 0x7fab440b5560 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab440af6d0 0x7fab440a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab440af6d0 0x7fab440a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44486/0 (socket says 172.21.15.7:44486) 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 -- 172.21.15.7:0/2655350238 learned_addr learned my addr 172.21.15.7:0/2655350238 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab495ba700 1 --1- 172.21.15.7:0/2655350238 >> v1:172.21.15.7:6789/0 conn(0x7fab440d0b20 0x7fab440b3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35712/0 (socket says 172.21.15.7:35712) 2023-10-06T11:22:35.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab48db9700 1 --2- 172.21.15.7:0/2655350238 >> v2:172.21.15.50:3300/0 conn(0x7fab440b5560 0x7fab440a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab437fe700 1 -- 172.21.15.7:0/2655350238 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585665249 0 0) 0x7fab440a94c0 con 0x7fab440d0b20 2023-10-06T11:22:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 -- 172.21.15.7:0/2655350238 >> v2:172.21.15.50:3300/0 conn(0x7fab440b5560 msgr2=0x7fab440a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 --2- 172.21.15.7:0/2655350238 >> v2:172.21.15.50:3300/0 conn(0x7fab440b5560 0x7fab440a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 -- 172.21.15.7:0/2655350238 >> v1:172.21.15.7:6789/0 conn(0x7fab440d0b20 legacy=0x7fab440b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 -- 172.21.15.7:0/2655350238 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440a8e70 con 0x7fab440af6d0 2023-10-06T11:22:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab43fff700 1 --2- 172.21.15.7:0/2655350238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440af6d0 0x7fab440a9e90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab4406eae0 tx=0x7fab34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a75405e25fc0c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab437fe700 1 -- 172.21.15.7:0/2655350238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34009e70 con 0x7fab440af6d0 2023-10-06T11:22:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab437fe700 1 -- 172.21.15.7:0/2655350238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3400b860 con 0x7fab440af6d0 2023-10-06T11:22:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.407+0000 7fab437fe700 1 -- 172.21.15.7:0/2655350238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400bc00 con 0x7fab440af6d0 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- 172.21.15.7:0/2655350238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440af6d0 msgr2=0x7fab440a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 --2- 172.21.15.7:0/2655350238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440af6d0 0x7fab440a9e90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab4406eae0 tx=0x7fab34009250 comp rx=0 tx=0).stop 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- 172.21.15.7:0/2655350238 shutdown_connections 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 --2- 172.21.15.7:0/2655350238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440af6d0 0x7fab440a9e90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 --2- 172.21.15.7:0/2655350238 >> v2:172.21.15.50:3300/0 conn(0x7fab440b5560 0x7fab440a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- 172.21.15.7:0/2655350238 >> 172.21.15.7:0/2655350238 conn(0x7fab440688c0 msgr2=0x7fab440cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- 172.21.15.7:0/2655350238 shutdown_connections 2023-10-06T11:22:35.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- 172.21.15.7:0/2655350238 wait complete. 2023-10-06T11:22:35.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 Processor -- start 2023-10-06T11:22:35.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.411+0000 7fab4b01d700 1 -- start start 2023-10-06T11:22:35.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab440af6d0 0x7fab44147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fab440b5560 0x7fab4415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415a7c0 con 0x7fab440d0b20 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44159bb0 con 0x7fab440af6d0 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab44159e90 con 0x7fab440b5560 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab43fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fab440b5560 0x7fab4415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44502/0 (socket says 172.21.15.7:44502) 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab48db9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab440af6d0 0x7fab44147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 -- 172.21.15.7:0/1745881921 learned_addr learned my addr 172.21.15.7:0/1745881921 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 -- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fab440b5560 msgr2=0x7fab4415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fab440b5560 0x7fab4415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 -- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab440af6d0 msgr2=0x7fab44147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab440af6d0 0x7fab44147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 -- 172.21.15.7:0/1745881921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340094f0 con 0x7fab440d0b20 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab495ba700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00ccc0 tx=0x7fab3c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c005df0 con 0x7fab440d0b20 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00a720 con 0x7fab440d0b20 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4415a170 con 0x7fab440d0b20 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00aae0 con 0x7fab440d0b20 2023-10-06T11:22:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab44147dd0 con 0x7fab440d0b20 2023-10-06T11:22:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.415+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4415a930 con 0x7fab440d0b20 2023-10-06T11:22:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.419+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fab3c0070f0 con 0x7fab440d0b20 2023-10-06T11:22:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.419+0000 7fab40ff9700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 0x7fab28062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.419+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fab3c07cd50 con 0x7fab440d0b20 2023-10-06T11:22:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.419+0000 7fab48db9700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 0x7fab28062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.419+0000 7fab48db9700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 0x7fab28062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009da0 tx=0x7fab2c009120 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.427+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fab3c04ced0 con 0x7fab440d0b20 2023-10-06T11:22:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.627+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab4404be00 con 0x7fab440d0b20 2023-10-06T11:22:35.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.627+0000 7fab40ff9700 1 -- 172.21.15.7:0/1745881921 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fab3c0504f0 con 0x7fab440d0b20 2023-10-06T11:22:35.633 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T11:22:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 msgr2=0x7fab28062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 0x7fab28062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009da0 tx=0x7fab2c009120 comp rx=0 tx=0).stop 2023-10-06T11:22:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 msgr2=0x7fab4415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00ccc0 tx=0x7fab3c001a40 comp rx=0 tx=0).stop 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 shutdown_connections 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fab28060550 0x7fab28062a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab440d0b20 0x7fab4415b190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fab440b5560 0x7fab4415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 --2- 172.21.15.7:0/1745881921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab440af6d0 0x7fab44147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 >> 172.21.15.7:0/1745881921 conn(0x7fab440688c0 msgr2=0x7fab440b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 shutdown_connections 2023-10-06T11:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:35.631+0000 7fab4b01d700 1 -- 172.21.15.7:0/1745881921 wait complete. 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:22:35.647 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:22:35.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.648 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-06T11:22:35.656 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.657 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.658 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:22:35.659 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42556sec preferred_lft 42556sec 2023-10-06T11:22:35.659 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2023-10-06T11:22:35.659 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.659 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:22:35.659 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 02:42:3c:4d:3d:a4 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 92:c1:94:25:78:23 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:22:35.660 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::5060:40ff:fec2:9946/64 scope link 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 92:c1:94:25:78:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::90c1:94ff:fe25:7823/64 scope link 2023-10-06T11:22:35.661 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.662 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:22:35.677 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-06T11:22:35.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.679 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-06T11:22:35.731 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-06T11:22:35.731 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:22:35.731 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:22:35.731 INFO:teuthology.orchestra.run.smithi050.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:22:35.731 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:22:35.732 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:22:35.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.732 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:35.732 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:35.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:22:35.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:22:35.733 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:22:35.733 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.774 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-06T11:22:35.781 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:22:35.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.782 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2023-10-06T11:22:35.829 INFO:teuthology.orchestra.run.smithi050.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T11:22:35.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.829 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:35.829 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:35.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:22:35.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T11:22:35.829 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:35.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.862 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:35.873 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether ee:e9:bb:d8:bf:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ece9:bbff:fed8:bfb5/64 scope link 2023-10-06T11:22:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:22:35.934 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether ee:e9:bb:d8:bf:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.935 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ece9:bbff:fed8:bfb5/64 scope link 2023-10-06T11:22:35.936 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:22:35.936 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T11:22:35.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T11:22:35.937 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:35.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:35.997 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-06T11:22:35.997 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-06T11:22:35.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.1 up 2023-10-06T11:22:35.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:22:35.997 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-06T11:22:36.028 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T11:22:36.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:36.028 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:36.076 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:22:36.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:22:36.077 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:36.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:36.096 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:36.096 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:36.107 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:36.107 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:36.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:36.108 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:36.113 INFO:teuthology.orchestra.run.smithi050.stdout:52 2023-10-06T11:22:36.114 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-06T11:22:36.114 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T11:22:36.116 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:36.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:36.134 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:36.192 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:36.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:36.193 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:36.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi050.stderr:2023-10-06T11:22:36.195+0000 7f4b7ba76380 -1 init, newargv = 0x561fe5a54a20 newargc=15 2023-10-06T11:22:36.199 INFO:teuthology.orchestra.run.smithi050.stdout:52 2023-10-06T11:22:36.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi050.stderr:ceph-fuse[39099]: starting ceph client 2023-10-06T11:22:36.207 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi050.stderr:ceph-fuse[39099]: starting fuse 2023-10-06T11:22:37.301 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-06T11:22:37.319 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:22:37.319 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:22:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:22:37.377 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:22:37.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.377 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-06T11:22:37.382 INFO:teuthology.orchestra.run.smithi050.stdout:52 2023-10-06T11:22:37.383 INFO:teuthology.orchestra.run.smithi050.stdout:53 2023-10-06T11:22:37.383 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T11:22:37.384 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:22:37.385 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.3")) 2023-10-06T11:22:37.386 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-06T11:22:37.469 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.3.39099.asok 2023-10-06T11:22:37.655 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39099.asok 2023-10-06T11:22:37.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.655 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39099.asok status 2023-10-06T11:22:37.797 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-06T11:22:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-06T11:22:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:22:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "3", 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "39099", 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4664, 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-06T11:22:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4664 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 553410160 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 553410160 2023-10-06T11:22:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4664 192.168.0.2:0/553410160", 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.2:0/553410160", 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 9, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 25, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-06T11:22:37.800 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-06T11:22:37.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.807 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-06T11:22:37.807 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:37.815 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-06T11:22:37.815 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:37.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:37.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.874 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-06T11:22:37.874 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:37.882 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-06T11:22:37.882 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:37.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:37.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.898 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-06T11:22:37.898 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:37.906 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-06T11:22:37.906 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:37.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:37.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.922 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-06T11:22:37.922 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:37.930 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-06T11:22:37.930 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:37.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:22:37.930 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:37.990 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T11:22:38.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:38.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 Processor -- start 2023-10-06T11:22:38.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- start start 2023-10-06T11:22:38.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b1815d0f0 0x7f7b18160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1806de10 0x7f7b180c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b181617c0 con 0x7f7b1815d4c0 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b18161aa0 con 0x7f7b1815d0f0 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18161d80 con 0x7f7b1806de10 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b1815d0f0 0x7f7b18160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b1815d0f0 0x7f7b18160ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44504/0 (socket says 172.21.15.7:44504) 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 -- 172.21.15.7:0/6673410 learned_addr learned my addr 172.21.15.7:0/6673410 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b16d9d700 1 --1- 172.21.15.7:0/6673410 >> v1:172.21.15.7:6789/0 conn(0x7f7b1815d4c0 0x7f7b180cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35728/0 (socket says 172.21.15.7:35728) 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b15d9b700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1806de10 0x7f7b180c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 -- 172.21.15.7:0/6673410 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1806de10 msgr2=0x7f7b180c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:38.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1806de10 0x7f7b180c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 -- 172.21.15.7:0/6673410 >> v1:172.21.15.7:6789/0 conn(0x7f7b1815d4c0 legacy=0x7f7b180cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 -- 172.21.15.7:0/6673410 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180ad220 con 0x7f7b1815d0f0 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1659c700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1815d0f0 0x7f7b18160ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b000099b0 tx=0x7f7b00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9f672823e0a9e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1559a700 1 -- 172.21.15.7:0/6673410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000b860 con 0x7f7b1815d0f0 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1559a700 1 -- 172.21.15.7:0/6673410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0000b9c0 con 0x7f7b1815d0f0 2023-10-06T11:22:38.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1559a700 1 -- 172.21.15.7:0/6673410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000bd60 con 0x7f7b1815d0f0 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- 172.21.15.7:0/6673410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1815d0f0 msgr2=0x7f7b18160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1815d0f0 0x7f7b18160ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b000099b0 tx=0x7f7b00009250 comp rx=0 tx=0).stop 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 -- 172.21.15.7:0/6673410 shutdown_connections 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.119+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1806de10 0x7f7b180c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/6673410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1815d0f0 0x7f7b18160ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 -- 172.21.15.7:0/6673410 >> 172.21.15.7:0/6673410 conn(0x7f7b18066840 msgr2=0x7f7b18066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 -- 172.21.15.7:0/6673410 shutdown_connections 2023-10-06T11:22:38.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 -- 172.21.15.7:0/6673410 wait complete. 2023-10-06T11:22:38.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 Processor -- start 2023-10-06T11:22:38.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.123+0000 7f7b1cc20700 1 -- start start 2023-10-06T11:22:38.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1815d0f0 0x7f7b180cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b1815d4c0 0x7f7b180cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1816d970 con 0x7f7b1806de10 2023-10-06T11:22:38.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1816e760 con 0x7f7b1815d4c0 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c7660 con 0x7f7b1815d0f0 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44508/0 (socket says 172.21.15.7:44508) 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 -- 172.21.15.7:0/3760410137 learned_addr learned my addr 172.21.15.7:0/3760410137 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b15d9b700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1815d0f0 0x7f7b180cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b16d9d700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b1815d4c0 0x7f7b180cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 -- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1815d0f0 msgr2=0x7f7b180cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1815d0f0 0x7f7b180cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 -- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b1815d4c0 msgr2=0x7f7b180cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b1815d4c0 0x7f7b180cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 -- 172.21.15.7:0/3760410137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000094f0 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1659c700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000b3c0 tx=0x7f7b00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00024440 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000245c0 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1816db50 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00024980 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.127+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1816e110 con 0x7f7b1806de10 2023-10-06T11:22:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1804be00 con 0x7f7b1806de10 2023-10-06T11:22:38.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7b0001a420 con 0x7f7b1806de10 2023-10-06T11:22:38.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b0e7fc700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 0x7f7af8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7b0007eb10 con 0x7f7b1806de10 2023-10-06T11:22:38.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b15d9b700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 0x7f7af8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.131+0000 7f7b15d9b700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 0x7f7af8062a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b1816b5a0 tx=0x7f7b08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.139+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b0004ec10 con 0x7f7b1806de10 2023-10-06T11:22:38.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.303+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b180ad500 con 0x7f7b1806de10 2023-10-06T11:22:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.303+0000 7f7b0e7fc700 1 -- 172.21.15.7:0/3760410137 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f7b00052230 con 0x7f7b1806de10 2023-10-06T11:22:38.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:38.309 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 msgr2=0x7f7af8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 0x7f7af8062a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b1816b5a0 tx=0x7f7b08008040 comp rx=0 tx=0).stop 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 msgr2=0x7f7b180c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000b3c0 tx=0x7f7b00017000 comp rx=0 tx=0).stop 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 shutdown_connections 2023-10-06T11:22:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7af8060580 0x7f7af8062a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b1815d4c0 0x7f7b180cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b1815d0f0 0x7f7b180cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 --2- 172.21.15.7:0/3760410137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1806de10 0x7f7b180c7120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 >> 172.21.15.7:0/3760410137 conn(0x7f7b18066840 msgr2=0x7f7b18157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 shutdown_connections 2023-10-06T11:22:38.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.307+0000 7f7b1cc20700 1 -- 172.21.15.7:0/3760410137 wait complete. 2023-10-06T11:22:38.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:38.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:38.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 Processor -- start 2023-10-06T11:22:38.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 -- start start 2023-10-06T11:22:38.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa4140ad3a0 0x7fa4140aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4140aef80 0x7fa414068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140acf10 con 0x7fa41406e920 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140b0900 con 0x7fa4140aef80 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4196f5700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140ac340 con 0x7fa4140ad3a0 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4140aef80 0x7fa414068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4140aef80 0x7fa414068060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44520/0 (socket says 172.21.15.7:44520) 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 -- 172.21.15.7:0/3286631520 learned_addr learned my addr 172.21.15.7:0/3286631520 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4127fc700 1 --2- 172.21.15.7:0/3286631520 >> v2:172.21.15.50:3300/0 conn(0x7fa4140ad3a0 0x7fa4140aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 -- 172.21.15.7:0/3286631520 >> v2:172.21.15.50:3300/0 conn(0x7fa4140ad3a0 msgr2=0x7fa4140aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 --2- 172.21.15.7:0/3286631520 >> v2:172.21.15.50:3300/0 conn(0x7fa4140ad3a0 0x7fa4140aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 -- 172.21.15.7:0/3286631520 >> v1:172.21.15.7:6789/0 conn(0x7fa41406e920 legacy=0x7fa41406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa411ffb700 1 -- 172.21.15.7:0/3286631520 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840211794 0 0) 0x7fa4140acf10 con 0x7fa41406e920 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 -- 172.21.15.7:0/3286631520 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140af420 con 0x7fa4140aef80 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.439+0000 7fa4137fe700 1 --2- 172.21.15.7:0/3286631520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140aef80 0x7fa414068060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa41406f430 tx=0x7fa40c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e2cde9e2f0a8ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa411ffb700 1 -- 172.21.15.7:0/3286631520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00e070 con 0x7fa4140aef80 2023-10-06T11:22:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa411ffb700 1 -- 172.21.15.7:0/3286631520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c00ba50 con 0x7fa4140aef80 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa411ffb700 1 -- 172.21.15.7:0/3286631520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0092e0 con 0x7fa4140aef80 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- 172.21.15.7:0/3286631520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140aef80 msgr2=0x7fa414068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- 172.21.15.7:0/3286631520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140aef80 0x7fa414068060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa41406f430 tx=0x7fa40c009840 comp rx=0 tx=0).stop 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- 172.21.15.7:0/3286631520 shutdown_connections 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- 172.21.15.7:0/3286631520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140aef80 0x7fa414068060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- 172.21.15.7:0/3286631520 >> v2:172.21.15.50:3300/0 conn(0x7fa4140ad3a0 0x7fa4140aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- 172.21.15.7:0/3286631520 >> 172.21.15.7:0/3286631520 conn(0x7fa414066840 msgr2=0x7fa414066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- 172.21.15.7:0/3286631520 shutdown_connections 2023-10-06T11:22:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- 172.21.15.7:0/3286631520 wait complete. 2023-10-06T11:22:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 Processor -- start 2023-10-06T11:22:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 -- start start 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa41406e920 0x7fa4141695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.443+0000 7fa4196f5700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa4140aef80 0x7fa4140a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140c8800 con 0x7fa4140ad3a0 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414169b10 con 0x7fa41406e920 2023-10-06T11:22:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140c7b80 con 0x7fa4140aef80 2023-10-06T11:22:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44524/0 (socket says 172.21.15.7:44524) 2023-10-06T11:22:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 -- 172.21.15.7:0/2731324236 learned_addr learned my addr 172.21.15.7:0/2731324236 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa412ffd700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa41406e920 0x7fa4141695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4137fe700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa4140aef80 0x7fa4140a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 -- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa4140aef80 msgr2=0x7fa4140a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa4140aef80 0x7fa4140a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 -- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa41406e920 msgr2=0x7fa4141695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa41406e920 0x7fa4141695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 -- 172.21.15.7:0/2731324236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c009ae0 con 0x7fa4140ad3a0 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4127fc700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa408006b00 tx=0x7fa408008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408011590 con 0x7fa4140ad3a0 2023-10-06T11:22:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080116f0 con 0x7fa4140ad3a0 2023-10-06T11:22:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408011ab0 con 0x7fa4140ad3a0 2023-10-06T11:22:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140c7e60 con 0x7fa4140ad3a0 2023-10-06T11:22:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140c8420 con 0x7fa4140ad3a0 2023-10-06T11:22:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa40801c460 con 0x7fa4140ad3a0 2023-10-06T11:22:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.447+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140c8940 con 0x7fa4140ad3a0 2023-10-06T11:22:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.451+0000 7fa3faffd700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 0x7fa400062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.451+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa40807cbc0 con 0x7fa4140ad3a0 2023-10-06T11:22:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.451+0000 7fa412ffd700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 0x7fa400062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.451+0000 7fa412ffd700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 0x7fa400062a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa414069470 tx=0x7fa404009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.459+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa4140c8940 con 0x7fa4140ad3a0 2023-10-06T11:22:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.615+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa4140c8940 con 0x7fa4140ad3a0 2023-10-06T11:22:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa3faffd700 1 -- 172.21.15.7:0/2731324236 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa4140c8940 con 0x7fa4140ad3a0 2023-10-06T11:22:38.622 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 msgr2=0x7fa400062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 0x7fa400062a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa414069470 tx=0x7fa404009040 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 msgr2=0x7fa4140c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa408006b00 tx=0x7fa408008040 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 shutdown_connections 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa400060580 0x7fa400062a30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa4140aef80 0x7fa4140a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140ad3a0 0x7fa4140c70c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 --2- 172.21.15.7:0/2731324236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa41406e920 0x7fa4141695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 >> 172.21.15.7:0/2731324236 conn(0x7fa414066840 msgr2=0x7fa4140b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 shutdown_connections 2023-10-06T11:22:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.619+0000 7fa4196f5700 1 -- 172.21.15.7:0/2731324236 wait complete. 2023-10-06T11:22:38.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.743+0000 7f5b2ce0c700 1 Processor -- start 2023-10-06T11:22:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.743+0000 7f5b2ce0c700 1 -- start start 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155940 0x7f5b281597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b2806e110 0x7f5b28156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b28156620 con 0x7f5b28155d10 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b28159ff0 con 0x7f5b2806e110 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2815a2d0 con 0x7f5b28155940 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b2806e110 0x7f5b28156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b26d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b28155d10 0x7f5b280c7f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35752/0 (socket says 172.21.15.7:35752) 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b26d9d700 1 -- 172.21.15.7:0/907057335 learned_addr learned my addr 172.21.15.7:0/907057335 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2659c700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155940 0x7f5b281597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 -- 172.21.15.7:0/907057335 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155940 msgr2=0x7f5b281597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155940 0x7f5b281597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 -- 172.21.15.7:0/907057335 >> v1:172.21.15.7:6789/0 conn(0x7f5b28155d10 legacy=0x7f5b280c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 -- 172.21.15.7:0/907057335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2815a5b0 con 0x7f5b2806e110 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b25d9b700 1 -- 172.21.15.7:0/907057335 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202300084 0 0) 0x7f5b28156620 con 0x7f5b28155d10 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b1ffff700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b28156ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b100098e0 tx=0x7f5b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeebfa41f17b314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b25d9b700 1 -- 172.21.15.7:0/907057335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000b640 con 0x7f5b2806e110 2023-10-06T11:22:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b25d9b700 1 -- 172.21.15.7:0/907057335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000b7a0 con 0x7f5b2806e110 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b25d9b700 1 -- 172.21.15.7:0/907057335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000bb40 con 0x7f5b2806e110 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/907057335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 msgr2=0x7f5b28156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b28156ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b100098e0 tx=0x7f5b100091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/907057335 shutdown_connections 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b28156ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/907057335 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155940 0x7f5b281597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/907057335 >> 172.21.15.7:0/907057335 conn(0x7f5b280cc2f0 msgr2=0x7f5b280cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/907057335 shutdown_connections 2023-10-06T11:22:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.747+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/907057335 wait complete. 2023-10-06T11:22:38.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 Processor -- start 2023-10-06T11:22:38.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 -- start start 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b28155940 0x7f5b28157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155d10 0x7f5b280c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2813cde0 con 0x7f5b2806e110 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2813e070 con 0x7f5b28155940 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2ce0c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2813d050 con 0x7f5b28155d10 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44552/0 (socket says 172.21.15.7:44552) 2023-10-06T11:22:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 -- 172.21.15.7:0/753407162 learned_addr learned my addr 172.21.15.7:0/753407162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:38.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b1ffff700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b28155940 0x7f5b28157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b26d9d700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155d10 0x7f5b280c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 -- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155d10 msgr2=0x7f5b280c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155d10 0x7f5b280c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 -- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b28155940 msgr2=0x7f5b28157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b28155940 0x7f5b28157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.751+0000 7f5b2659c700 1 -- 172.21.15.7:0/753407162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009420 con 0x7f5b2806e110 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b2659c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800ec80 tx=0x7f5b180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800c850 con 0x7f5b2806e110 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1800c9b0 con 0x7f5b2806e110 2023-10-06T11:22:38.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2813d330 con 0x7f5b2806e110 2023-10-06T11:22:38.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800cd50 con 0x7f5b2806e110 2023-10-06T11:22:38.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b28158260 con 0x7f5b2806e110 2023-10-06T11:22:38.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2804be00 con 0x7f5b2806e110 2023-10-06T11:22:38.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5b18013680 con 0x7f5b2806e110 2023-10-06T11:22:38.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 0x7f5b08062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:38.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5b1807cec0 con 0x7f5b2806e110 2023-10-06T11:22:38.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.755+0000 7f5b1ffff700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 0x7f5b08062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:38.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.759+0000 7f5b1ffff700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 0x7f5b08062a00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b100093f0 tx=0x7f5b10017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.767+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5b1804d060 con 0x7f5b2806e110 2023-10-06T11:22:38.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b281562b0 con 0x7f5b2806e110 2023-10-06T11:22:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b1e7fc700 1 -- 172.21.15.7:0/753407162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b18050680 con 0x7f5b2806e110 2023-10-06T11:22:38.935 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:38.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 msgr2=0x7f5b08062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 0x7f5b08062a00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b100093f0 tx=0x7f5b10017000 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 msgr2=0x7f5b2813db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800ec80 tx=0x7f5b180091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 shutdown_connections 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5b08060550 0x7f5b08062a00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5b28155d10 0x7f5b280c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b28155940 0x7f5b28157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 --2- 172.21.15.7:0/753407162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806e110 0x7f5b2813db30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 >> 172.21.15.7:0/753407162 conn(0x7f5b280cc2f0 msgr2=0x7f5b2806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 shutdown_connections 2023-10-06T11:22:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:38.931+0000 7f5b2ce0c700 1 -- 172.21.15.7:0/753407162 wait complete. 2023-10-06T11:22:38.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.059+0000 7fada759e700 1 Processor -- start 2023-10-06T11:22:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.059+0000 7fada759e700 1 -- start start 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fada80cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80cc3b0 con 0x7fada806d6e0 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada759e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80cc690 con 0x7fada80ac9f0 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada8157320 con 0x7fada80cd450 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fada806d6e0 0x7fada80cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35758/0 (socket says 172.21.15.7:35758) 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 -- 172.21.15.7:0/4102709337 learned_addr learned my addr 172.21.15.7:0/4102709337 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada6d9d700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fada80cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946657414 0 0) 0x7fada80cc3b0 con 0x7fada806d6e0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad94003570 con 0x7fada806d6e0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682143518 0 0) 0x7fada80cc690 con 0x7fada80ac9f0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada80cc3b0 con 0x7fada80ac9f0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358909542 0 0) 0x7fad94003570 con 0x7fada806d6e0 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 msgr2=0x7fada80cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:39.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fada80cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 >> v1:172.21.15.50:6789/0 conn(0x7fada80ac9f0 legacy=0x7fada806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80cc970 con 0x7fada806d6e0 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7fad90001470 con 0x7fada806d6e0 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 >> v1:172.21.15.7:6789/0 conn(0x7fada806d6e0 legacy=0x7fada80cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fad94003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fad94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad94004d50 0x7fad940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9400bb70 con 0x7fad94004180 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9400bce0 con 0x7fad94004d50 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9400bea0 con 0x7fada80cd450 2023-10-06T11:22:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fad94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada659c700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fad94003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada6d9d700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad94004d50 0x7fad940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 -- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 msgr2=0x7fad94003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada80cd450 0x7fad94003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 -- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad94004d50 msgr2=0x7fad940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad94004d50 0x7fad940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 -- 172.21.15.7:0/4102709337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94003570 con 0x7fad94004180 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada5d9b700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fad94004810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fada80cb650 tx=0x7fad90001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9153d1dde70253 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000cb60 con 0x7fad94004180 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9000ccc0 con 0x7fad94004180 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.063+0000 7fada559a700 1 -- 172.21.15.7:0/4102709337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90005b70 con 0x7fad94004180 2023-10-06T11:22:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- 172.21.15.7:0/4102709337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 msgr2=0x7fad94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 --2- 172.21.15.7:0/4102709337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fad94004810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fada80cb650 tx=0x7fad90001e10 comp rx=0 tx=0).stop 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada6d9d700 1 -- 172.21.15.7:0/4102709337 reap_dead start 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- 172.21.15.7:0/4102709337 shutdown_connections 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- 172.21.15.7:0/4102709337 >> 172.21.15.7:0/4102709337 conn(0x7fada80ce570 msgr2=0x7fada80ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- 172.21.15.7:0/4102709337 shutdown_connections 2023-10-06T11:22:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- 172.21.15.7:0/4102709337 wait complete. 2023-10-06T11:22:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 Processor -- start 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.067+0000 7fada759e700 1 -- start start 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fada815c100 0x7fada815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada815e050 0x7fada80b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815e7d0 con 0x7fad94004180 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80b0fd0 con 0x7fada815c100 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada80b12b0 con 0x7fada815e050 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44566/0 (socket says 172.21.15.7:44566) 2023-10-06T11:22:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 -- 172.21.15.7:0/2935118388 learned_addr learned my addr 172.21.15.7:0/2935118388 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada6d9d700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada815e050 0x7fada80b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada5d9b700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fada815c100 0x7fada815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 -- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada815e050 msgr2=0x7fada80b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada815e050 0x7fada80b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 -- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fada815c100 msgr2=0x7fada815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fada815c100 0x7fada815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 -- 172.21.15.7:0/2935118388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80b1590 con 0x7fad94004180 2023-10-06T11:22:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada659c700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad98002b50 tx=0x7fad98007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800f8d0 con 0x7fad94004180 2023-10-06T11:22:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada80b1870 con 0x7fad94004180 2023-10-06T11:22:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9800fa50 con 0x7fad94004180 2023-10-06T11:22:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9800fe10 con 0x7fad94004180 2023-10-06T11:22:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.071+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada80b22d0 con 0x7fad94004180 2023-10-06T11:22:39.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fad98013600 con 0x7fad94004180 2023-10-06T11:22:39.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada806eb50 con 0x7fad94004180 2023-10-06T11:22:39.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fad9e7fc700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 0x7fad88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fad9807d390 con 0x7fad94004180 2023-10-06T11:22:39.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fada5d9b700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 0x7fad88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.075+0000 7fada5d9b700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 0x7fad88062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad90008fd0 tx=0x7fad90015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.083+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fad9804d490 con 0x7fad94004180 2023-10-06T11:22:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.247+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fada815e910 con 0x7fad94004180 2023-10-06T11:22:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.247+0000 7fad9e7fc700 1 -- 172.21.15.7:0/2935118388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad98050ab0 con 0x7fad94004180 2023-10-06T11:22:39.252 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:39.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 msgr2=0x7fad88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 0x7fad88062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad90008fd0 tx=0x7fad90015000 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 msgr2=0x7fada815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad98002b50 tx=0x7fad98007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 shutdown_connections 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fad88060550 0x7fad88062a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fada815e050 0x7fada80b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fada815c100 0x7fada815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 --2- 172.21.15.7:0/2935118388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad94004180 0x7fada815bbc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 >> 172.21.15.7:0/2935118388 conn(0x7fada80ce570 msgr2=0x7fada815b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 shutdown_connections 2023-10-06T11:22:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.251+0000 7fada759e700 1 -- 172.21.15.7:0/2935118388 wait complete. 2023-10-06T11:22:39.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:39.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.371+0000 7f208e1c4700 1 Processor -- start 2023-10-06T11:22:39.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.371+0000 7f208e1c4700 1 -- start start 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f208806c1c0 0x7f208815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20881598e0 con 0x7f208815c980 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880acf50 con 0x7f20880acb80 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2088159380 con 0x7f208806c1c0 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f208806c1c0 0x7f208815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f208806c1c0 0x7f208815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44580/0 (socket says 172.21.15.7:44580) 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 -- 172.21.15.7:0/1510984351 learned_addr learned my addr 172.21.15.7:0/1510984351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208c9c1700 1 --1- 172.21.15.7:0/1510984351 >> v1:172.21.15.7:6789/0 conn(0x7f208815c980 0x7f208806bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35762/0 (socket says 172.21.15.7:35762) 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 -- 172.21.15.7:0/1510984351 >> v1:172.21.15.50:6789/0 conn(0x7f20880acb80 legacy=0x7f2088159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 -- 172.21.15.7:0/1510984351 >> v1:172.21.15.7:6789/0 conn(0x7f208815c980 legacy=0x7f208806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f207ffff700 1 -- 172.21.15.7:0/1510984351 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151466520 0 0) 0x7f20881598e0 con 0x7f208815c980 2023-10-06T11:22:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 -- 172.21.15.7:0/1510984351 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208815d400 con 0x7f208806c1c0 2023-10-06T11:22:39.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208d9c3700 1 --2- 172.21.15.7:0/1510984351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208806c1c0 0x7f208815ce20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20800099b0 tx=0x7f2080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be2d767e3ce4362 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f207ffff700 1 -- 172.21.15.7:0/1510984351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000b860 con 0x7f208806c1c0 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f207ffff700 1 -- 172.21.15.7:0/1510984351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208000b9c0 con 0x7f208806c1c0 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f207ffff700 1 -- 172.21.15.7:0/1510984351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000bd60 con 0x7f208806c1c0 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- 172.21.15.7:0/1510984351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208806c1c0 msgr2=0x7f208815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1510984351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208806c1c0 0x7f208815ce20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20800099b0 tx=0x7f2080009250 comp rx=0 tx=0).stop 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- 172.21.15.7:0/1510984351 shutdown_connections 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1510984351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208806c1c0 0x7f208815ce20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- 172.21.15.7:0/1510984351 >> 172.21.15.7:0/1510984351 conn(0x7f20880cdb00 msgr2=0x7f20880cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- 172.21.15.7:0/1510984351 shutdown_connections 2023-10-06T11:22:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.375+0000 7f208e1c4700 1 -- 172.21.15.7:0/1510984351 wait complete. 2023-10-06T11:22:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 Processor -- start 2023-10-06T11:22:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- start start 2023-10-06T11:22:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f208806c1c0 0x7f208816c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f20880acb80 0x7f2088169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880cc620 con 0x7f208815c980 2023-10-06T11:22:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208816c8f0 con 0x7f208806c1c0 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208816a500 con 0x7f20880acb80 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44590/0 (socket says 172.21.15.7:44590) 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 -- 172.21.15.7:0/1627267653 learned_addr learned my addr 172.21.15.7:0/1627267653 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d1c2700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f208806c1c0 0x7f208816c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208c9c1700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f20880acb80 0x7f2088169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 -- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f20880acb80 msgr2=0x7f2088169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f20880acb80 0x7f2088169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 -- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f208806c1c0 msgr2=0x7f208816c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f208806c1c0 0x7f208816c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 -- 172.21.15.7:0/1627267653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800094f0 con 0x7f208815c980 2023-10-06T11:22:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208d9c3700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2080006f50 tx=0x7f2080017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208001f070 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080006bc0 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20880c81d0 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080011ea0 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.379+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20880c8790 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.383+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208816a7a0 con 0x7f208815c980 2023-10-06T11:22:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.383+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f208001aa70 con 0x7f208815c980 2023-10-06T11:22:39.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.383+0000 7f207d7fa700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 0x7f206c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.383+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f208007e060 con 0x7f208815c980 2023-10-06T11:22:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.383+0000 7f208d1c2700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 0x7f206c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.387+0000 7f208d1c2700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 0x7f206c062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2084007fd0 tx=0x7f2084009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.391+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f208816a7a0 con 0x7f208815c980 2023-10-06T11:22:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.559+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f208816a7a0 con 0x7f208815c980 2023-10-06T11:22:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.559+0000 7f207d7fa700 1 -- 172.21.15.7:0/1627267653 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f208816a7a0 con 0x7f208815c980 2023-10-06T11:22:39.564 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 msgr2=0x7f206c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 0x7f206c062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2084007fd0 tx=0x7f2084009040 comp rx=0 tx=0).stop 2023-10-06T11:22:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 msgr2=0x7f20880cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2080006f50 tx=0x7f2080017000 comp rx=0 tx=0).stop 2023-10-06T11:22:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 shutdown_connections 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f206c060550 0x7f206c062a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208815c980 0x7f20880cded0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f20880acb80 0x7f2088169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 --2- 172.21.15.7:0/1627267653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f208806c1c0 0x7f208816c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 >> 172.21.15.7:0/1627267653 conn(0x7f20880cdb00 msgr2=0x7f208815b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 shutdown_connections 2023-10-06T11:22:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.563+0000 7f208e1c4700 1 -- 172.21.15.7:0/1627267653 wait complete. 2023-10-06T11:22:39.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 Processor -- start 2023-10-06T11:22:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 -- start start 2023-10-06T11:22:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f23dc15c3e0 0x7f23dc0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23dc06d450 0x7f23dc15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0c84f0 con 0x7f23dc06cfd0 2023-10-06T11:22:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc0cca80 con 0x7f23dc06d450 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e3567700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc0c9970 con 0x7f23dc15c3e0 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e2d66700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23dc06d450 0x7f23dc15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e2d66700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23dc06d450 0x7f23dc15d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44604/0 (socket says 172.21.15.7:44604) 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e2d66700 1 -- 172.21.15.7:0/2903715070 learned_addr learned my addr 172.21.15.7:0/2903715070 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e1d64700 1 --1- 172.21.15.7:0/2903715070 >> v1:172.21.15.7:6789/0 conn(0x7f23dc06cfd0 0x7f23dc15ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35772/0 (socket says 172.21.15.7:35772) 2023-10-06T11:22:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.687+0000 7f23e2565700 1 --2- 172.21.15.7:0/2903715070 >> v2:172.21.15.50:3300/0 conn(0x7f23dc15c3e0 0x7f23dc0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e2d66700 1 -- 172.21.15.7:0/2903715070 >> v2:172.21.15.50:3300/0 conn(0x7f23dc15c3e0 msgr2=0x7f23dc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:39.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e2d66700 1 --2- 172.21.15.7:0/2903715070 >> v2:172.21.15.50:3300/0 conn(0x7f23dc15c3e0 0x7f23dc0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e1563700 1 -- 172.21.15.7:0/2903715070 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941338033 0 0) 0x7f23dc0c84f0 con 0x7f23dc06cfd0 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e2d66700 1 -- 172.21.15.7:0/2903715070 >> v1:172.21.15.7:6789/0 conn(0x7f23dc06cfd0 legacy=0x7f23dc15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e2d66700 1 -- 172.21.15.7:0/2903715070 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc15c7b0 con 0x7f23dc06d450 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e2d66700 1 --2- 172.21.15.7:0/2903715070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15d130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23dc071140 tx=0x7f23d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f873699ee0ae1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e1563700 1 -- 172.21.15.7:0/2903715070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4009e70 con 0x7f23dc06d450 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e1563700 1 -- 172.21.15.7:0/2903715070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d400b860 con 0x7f23dc06d450 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e1563700 1 -- 172.21.15.7:0/2903715070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d400bc00 con 0x7f23dc06d450 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- 172.21.15.7:0/2903715070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 msgr2=0x7f23dc15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 --2- 172.21.15.7:0/2903715070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15d130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23dc071140 tx=0x7f23d4009250 comp rx=0 tx=0).stop 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- 172.21.15.7:0/2903715070 shutdown_connections 2023-10-06T11:22:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 --2- 172.21.15.7:0/2903715070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15d130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 --2- 172.21.15.7:0/2903715070 >> v2:172.21.15.50:3300/0 conn(0x7f23dc15c3e0 0x7f23dc0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- 172.21.15.7:0/2903715070 >> 172.21.15.7:0/2903715070 conn(0x7f23dc0ce570 msgr2=0x7f23dc0ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- 172.21.15.7:0/2903715070 shutdown_connections 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- 172.21.15.7:0/2903715070 wait complete. 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 Processor -- start 2023-10-06T11:22:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.691+0000 7f23e3567700 1 -- start start 2023-10-06T11:22:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23dc06cfd0 0x7f23dc1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 0x7f23dc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc15e750 con 0x7f23dc06d450 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc069a10 con 0x7f23dc15c3e0 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0690e0 con 0x7f23dc06cfd0 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44608/0 (socket says 172.21.15.7:44608) 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 -- 172.21.15.7:0/1025813987 learned_addr learned my addr 172.21.15.7:0/1025813987 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e2565700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23dc06cfd0 0x7f23dc1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e2d66700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 0x7f23dc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 -- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23dc06cfd0 msgr2=0x7f23dc1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23dc06cfd0 0x7f23dc1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 -- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 msgr2=0x7f23dc069ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 0x7f23dc069ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 -- 172.21.15.7:0/1025813987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40094f0 con 0x7f23dc06d450 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e2d66700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 0x7f23dc069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e1d64700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23d800ce00 tx=0x7f23d8001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8005df0 con 0x7f23dc06d450 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800a920 con 0x7f23dc06d450 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0693c0 con 0x7f23dc06d450 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800ace0 con 0x7f23dc06d450 2023-10-06T11:22:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.695+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc0b34f0 con 0x7f23dc06d450 2023-10-06T11:22:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc069ba0 con 0x7f23dc06d450 2023-10-06T11:22:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f23d8007300 con 0x7f23dc06d450 2023-10-06T11:22:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23d27fc700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 0x7f23c8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f23d807ceb0 con 0x7f23dc06d450 2023-10-06T11:22:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23e2565700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 0x7f23c8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.699+0000 7f23e2565700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 0x7f23c8062a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d10 tx=0x7f23cc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.707+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f23d804d0e0 con 0x7f23dc06d450 2023-10-06T11:22:39.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.867+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23dc0c2eb0 con 0x7f23dc06d450 2023-10-06T11:22:39.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23d27fc700 1 -- 172.21.15.7:0/1025813987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f23d8050700 con 0x7f23dc06d450 2023-10-06T11:22:39.874 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 msgr2=0x7f23c8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 0x7f23c8062a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d10 tx=0x7f23cc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 msgr2=0x7f23dc15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f23d800ce00 tx=0x7f23d8001bc0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 shutdown_connections 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23c8060580 0x7f23c8062a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23dc15c3e0 0x7f23dc069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23dc06d450 0x7f23dc15dca0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 --2- 172.21.15.7:0/1025813987 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23dc06cfd0 0x7f23dc1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:39.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 >> 172.21.15.7:0/1025813987 conn(0x7f23dc0ce570 msgr2=0x7f23dc0c3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:39.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 shutdown_connections 2023-10-06T11:22:39.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.871+0000 7f23e3567700 1 -- 172.21.15.7:0/1025813987 wait complete. 2023-10-06T11:22:39.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 Processor -- start 2023-10-06T11:22:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- start start 2023-10-06T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8084155840 0x7f8084159670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840c8c20 con 0x7f808406c0b0 2023-10-06T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840c8610 con 0x7f8084159bb0 2023-10-06T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808415a610 con 0x7f8084155840 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8084155840 0x7f8084159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8084155840 0x7f8084159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44614/0 (socket says 172.21.15.7:44614) 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 -- 172.21.15.7:0/2928564341 learned_addr learned my addr 172.21.15.7:0/2928564341 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f80899a7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f808406c0b0 0x7f80840cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35776/0 (socket says 172.21.15.7:35776) 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 -- 172.21.15.7:0/2928564341 >> v1:172.21.15.50:6789/0 conn(0x7f8084159bb0 legacy=0x7f80840c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 -- 172.21.15.7:0/2928564341 >> v1:172.21.15.7:6789/0 conn(0x7f808406c0b0 legacy=0x7f80840cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 -- 172.21.15.7:0/2928564341 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808415a050 con 0x7f8084155840 2023-10-06T11:22:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808a1a8700 1 --2- 172.21.15.7:0/2928564341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8084155840 0x7f8084159670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f808000d1a0 tx=0x7f808000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf3ddae28f0cc7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f80891a6700 1 -- 172.21.15.7:0/2928564341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080010070 con 0x7f8084155840 2023-10-06T11:22:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f80891a6700 1 -- 172.21.15.7:0/2928564341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000de80 con 0x7f8084155840 2023-10-06T11:22:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f80891a6700 1 -- 172.21.15.7:0/2928564341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000b320 con 0x7f8084155840 2023-10-06T11:22:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- 172.21.15.7:0/2928564341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8084155840 msgr2=0x7f8084159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 --2- 172.21.15.7:0/2928564341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8084155840 0x7f8084159670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f808000d1a0 tx=0x7f808000ba50 comp rx=0 tx=0).stop 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- 172.21.15.7:0/2928564341 shutdown_connections 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 --2- 172.21.15.7:0/2928564341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8084155840 0x7f8084159670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- 172.21.15.7:0/2928564341 >> 172.21.15.7:0/2928564341 conn(0x7f808406f380 msgr2=0x7f808406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- 172.21.15.7:0/2928564341 shutdown_connections 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.995+0000 7f808b1aa700 1 -- 172.21.15.7:0/2928564341 wait complete. 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 Processor -- start 2023-10-06T11:22:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- start start 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8084155840 0x7f80840c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 0x7f80841652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840c4540 con 0x7f80841657e0 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084166180 con 0x7f8084155840 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084158570 con 0x7f8084159bb0 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44628/0 (socket says 172.21.15.7:44628) 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 -- 172.21.15.7:0/345643724 learned_addr learned my addr 172.21.15.7:0/345643724 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a1a8700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8084155840 0x7f80840c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f80899a7700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 0x7f80841652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 -- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 msgr2=0x7f80841652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 0x7f80841652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 -- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8084155840 msgr2=0x7f80840c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8084155840 0x7f80840c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 -- 172.21.15.7:0/345643724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808000bcf0 con 0x7f80841657e0 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f80899a7700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 0x7f80841652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808a9a9700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f807c00b5e0 tx=0x7f807c009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f510 con 0x7f80841657e0 2023-10-06T11:22:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c00f690 con 0x7f80841657e0 2023-10-06T11:22:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8084150e60 con 0x7f80841657e0 2023-10-06T11:22:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00fa50 con 0x7f80841657e0 2023-10-06T11:22:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:39.999+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8084157c40 con 0x7f80841657e0 2023-10-06T11:22:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f807c01a400 con 0x7f80841657e0 2023-10-06T11:22:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808406db40 con 0x7f80841657e0 2023-10-06T11:22:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f807a7fc700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 0x7f8068062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f807c07c650 con 0x7f80841657e0 2023-10-06T11:22:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f808a1a8700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 0x7f8068062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.003+0000 7f808a1a8700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 0x7f8068062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f808000ba90 tx=0x7f8080019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.011+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f807c04c870 con 0x7f80841657e0 2023-10-06T11:22:40.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.171+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f808404b5c0 con 0x7f80841657e0 2023-10-06T11:22:40.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.171+0000 7f807a7fc700 1 -- 172.21.15.7:0/345643724 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f807c04fe90 con 0x7f80841657e0 2023-10-06T11:22:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:40.176 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 msgr2=0x7f8068062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 0x7f8068062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f808000ba90 tx=0x7f8080019000 comp rx=0 tx=0).stop 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 msgr2=0x7f8084165c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f807c00b5e0 tx=0x7f807c009060 comp rx=0 tx=0).stop 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 shutdown_connections 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8068060550 0x7f8068062a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80841657e0 0x7f8084165c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8084159bb0 0x7f80841652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 --2- 172.21.15.7:0/345643724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8084155840 0x7f80840c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 >> 172.21.15.7:0/345643724 conn(0x7f808406f380 msgr2=0x7f8084154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 shutdown_connections 2023-10-06T11:22:40.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.175+0000 7f808b1aa700 1 -- 172.21.15.7:0/345643724 wait complete. 2023-10-06T11:22:40.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:40.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:40.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.295+0000 7f913759e700 1 Processor -- start 2023-10-06T11:22:40.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.295+0000 7f913759e700 1 -- start start 2023-10-06T11:22:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9138156090 0x7f9138156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91381569a0 0x7f91380acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91381571f0 con 0x7f91380ad4a0 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9138157b00 con 0x7f9138156090 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91380adcd0 con 0x7f91381569a0 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f9136d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91380ad4a0 0x7f9138157f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35778/0 (socket says 172.21.15.7:35778) 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f9136d9d700 1 -- 172.21.15.7:0/4091974437 learned_addr learned my addr 172.21.15.7:0/4091974437 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 --2- 172.21.15.7:0/4091974437 >> v2:172.21.15.50:3300/0 conn(0x7f9138156090 0x7f9138156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f9135d9b700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91381569a0 0x7f91380acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913559a700 1 -- 172.21.15.7:0/4091974437 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614708850 0 0) 0x7f91381571f0 con 0x7f91380ad4a0 2023-10-06T11:22:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913559a700 1 -- 172.21.15.7:0/4091974437 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9124003570 con 0x7f91380ad4a0 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 -- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91381569a0 msgr2=0x7f91380acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91381569a0 0x7f91380acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 -- 172.21.15.7:0/4091974437 >> v1:172.21.15.7:6789/0 conn(0x7f91380ad4a0 legacy=0x7f9138157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 -- 172.21.15.7:0/4091974437 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138157440 con 0x7f9138156090 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913659c700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138156460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91300098e0 tx=0x7f91300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba82880b355c0254 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913559a700 1 -- 172.21.15.7:0/4091974437 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130011630 con 0x7f9138156090 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913559a700 1 -- 172.21.15.7:0/4091974437 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130011790 con 0x7f9138156090 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913559a700 1 -- 172.21.15.7:0/4091974437 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130011b10 con 0x7f9138156090 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 msgr2=0x7f9138156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138156460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91300098e0 tx=0x7f91300091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- 172.21.15.7:0/4091974437 shutdown_connections 2023-10-06T11:22:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91381569a0 0x7f91380acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 --2- 172.21.15.7:0/4091974437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138156460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- 172.21.15.7:0/4091974437 >> 172.21.15.7:0/4091974437 conn(0x7f91380cc3c0 msgr2=0x7f9138067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- 172.21.15.7:0/4091974437 shutdown_connections 2023-10-06T11:22:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.299+0000 7f913759e700 1 -- 172.21.15.7:0/4091974437 wait complete. 2023-10-06T11:22:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 Processor -- start 2023-10-06T11:22:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- start start 2023-10-06T11:22:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91380ad4a0 0x7f913814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913814a680 con 0x7f91381569a0 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9138169ec0 con 0x7f9138156090 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913814b620 con 0x7f91380ad4a0 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44644/0 (socket says 172.21.15.7:44644) 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 -- 172.21.15.7:0/1323727668 learned_addr learned my addr 172.21.15.7:0/1323727668 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913659c700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91380ad4a0 0x7f913814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9135d9b700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 -- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91380ad4a0 msgr2=0x7f913814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91380ad4a0 0x7f913814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 -- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 msgr2=0x7f9138169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 -- 172.21.15.7:0/1323727668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130009420 con 0x7f91381569a0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f9136d9d700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9128001460 tx=0x7f9128001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800a890 con 0x7f91381569a0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912800a9f0 con 0x7f91381569a0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800ad70 con 0x7f91381569a0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913814b900 con 0x7f91381569a0 2023-10-06T11:22:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.303+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913814bec0 con 0x7f91381569a0 2023-10-06T11:22:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91380a88b0 con 0x7f91381569a0 2023-10-06T11:22:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9128007300 con 0x7f91381569a0 2023-10-06T11:22:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f912e7fc700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 0x7f9118062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f912807cff0 con 0x7f91381569a0 2023-10-06T11:22:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f913659c700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 0x7f9118062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.307+0000 7f913659c700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 0x7f9118062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91300091e0 tx=0x7f9130017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.311+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f912804d180 con 0x7f91381569a0 2023-10-06T11:22:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9138157720 con 0x7f91381569a0 2023-10-06T11:22:40.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f912e7fc700 1 -- 172.21.15.7:0/1323727668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91280507a0 con 0x7f91381569a0 2023-10-06T11:22:40.526 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 msgr2=0x7f9118062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 0x7f9118062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91300091e0 tx=0x7f9130017000 comp rx=0 tx=0).stop 2023-10-06T11:22:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 msgr2=0x7f91380aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.479+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9128001460 tx=0x7f9128001ae0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 shutdown_connections 2023-10-06T11:22:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9118060550 0x7f9118062a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91381569a0 0x7f91380aff70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9138156090 0x7f9138169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 --2- 172.21.15.7:0/1323727668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f91380ad4a0 0x7f913814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 >> 172.21.15.7:0/1323727668 conn(0x7f91380cc3c0 msgr2=0x7f9138154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 shutdown_connections 2023-10-06T11:22:40.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.483+0000 7f913759e700 1 -- 172.21.15.7:0/1323727668 wait complete. 2023-10-06T11:22:40.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:40.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 Processor -- start 2023-10-06T11:22:40.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 -- start start 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3780ce260 0x7fa378159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37815d320 con 0x7fa3780ccdb0 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3780ce6a0 con 0x7fa3780b21d0 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.643+0000 7fa37f2ae700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37815a460 con 0x7fa3780ce260 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3780ce260 0x7fa378159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3780ce260 0x7fa378159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44660/0 (socket says 172.21.15.7:44660) 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37daab700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3780ccdb0 0x7fa3780ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35780/0 (socket says 172.21.15.7:35780) 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 -- 172.21.15.7:0/1701322902 learned_addr learned my addr 172.21.15.7:0/1701322902 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 -- 172.21.15.7:0/1701322902 >> v1:172.21.15.50:6789/0 conn(0x7fa3780b21d0 legacy=0x7fa3780cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 -- 172.21.15.7:0/1701322902 >> v1:172.21.15.7:6789/0 conn(0x7fa3780ccdb0 legacy=0x7fa3780ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 -- 172.21.15.7:0/1701322902 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780ce940 con 0x7fa3780ce260 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37eaad700 1 --2- 172.21.15.7:0/1701322902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ce260 0x7fa378159f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa370009f20 tx=0x7fa370009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2a6c7e19cbecdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37d2aa700 1 -- 172.21.15.7:0/1701322902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000e070 con 0x7fa3780ce260 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37d2aa700 1 -- 172.21.15.7:0/1701322902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37000ba80 con 0x7fa3780ce260 2023-10-06T11:22:40.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37d2aa700 1 -- 172.21.15.7:0/1701322902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700092e0 con 0x7fa3780ce260 2023-10-06T11:22:40.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 -- 172.21.15.7:0/1701322902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ce260 msgr2=0x7fa378159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/1701322902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ce260 0x7fa378159f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa370009f20 tx=0x7fa370009840 comp rx=0 tx=0).stop 2023-10-06T11:22:40.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 -- 172.21.15.7:0/1701322902 shutdown_connections 2023-10-06T11:22:40.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/1701322902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ce260 0x7fa378159f20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 -- 172.21.15.7:0/1701322902 >> 172.21.15.7:0/1701322902 conn(0x7fa378067e30 msgr2=0x7fa378068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 -- 172.21.15.7:0/1701322902 shutdown_connections 2023-10-06T11:22:40.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.647+0000 7fa37f2ae700 1 -- 172.21.15.7:0/1701322902 wait complete. 2023-10-06T11:22:40.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 Processor -- start 2023-10-06T11:22:40.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 -- start start 2023-10-06T11:22:40.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3780b21d0 0x7fa37814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 0x7fa37806a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780c83b0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780c77a0 con 0x7fa3780b21d0 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37f2ae700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780c7a80 con 0x7fa3780ce260 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37daab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37e2ac700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3780b21d0 0x7fa37814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37daab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44672/0 (socket says 172.21.15.7:44672) 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37daab700 1 -- 172.21.15.7:0/3652074585 learned_addr learned my addr 172.21.15.7:0/3652074585 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.651+0000 7fa37eaad700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 0x7fa37806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 -- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 msgr2=0x7fa37806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 0x7fa37806a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 -- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3780b21d0 msgr2=0x7fa37814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3780b21d0 0x7fa37814d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 -- 172.21.15.7:0/3652074585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370009ae0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37eaad700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 0x7fa37806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37daab700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c7c0 tx=0x7fa374007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374005a70 con 0x7fa3780ccdb0 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374005bd0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3780c7d60 con 0x7fa3780ccdb0 2023-10-06T11:22:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740070d0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37814dc50 con 0x7fa3780ccdb0 2023-10-06T11:22:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.655+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37806cc90 con 0x7fa3780ccdb0 2023-10-06T11:22:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.659+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa374007480 con 0x7fa3780ccdb0 2023-10-06T11:22:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.659+0000 7fa36a7fc700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 0x7fa35c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.659+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa37407c150 con 0x7fa3780ccdb0 2023-10-06T11:22:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.659+0000 7fa37e2ac700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 0x7fa35c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.659+0000 7fa37e2ac700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 0x7fa35c062a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa36c003a80 tx=0x7fa36c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.667+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa37404c2f0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.831+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa37804b5c0 con 0x7fa3780ccdb0 2023-10-06T11:22:40.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.831+0000 7fa36a7fc700 1 -- 172.21.15.7:0/3652074585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa37404f910 con 0x7fa3780ccdb0 2023-10-06T11:22:40.837 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 msgr2=0x7fa35c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 0x7fa35c062a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa36c003a80 tx=0x7fa36c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 msgr2=0x7fa3780c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c7c0 tx=0x7fa374007f70 comp rx=0 tx=0).stop 2023-10-06T11:22:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 shutdown_connections 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa35c060580 0x7fa35c062a30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa3780ce260 0x7fa37806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3780ccdb0 0x7fa3780c6d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 --2- 172.21.15.7:0/3652074585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3780b21d0 0x7fa37814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 >> 172.21.15.7:0/3652074585 conn(0x7fa378067e30 msgr2=0x7fa3780cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 shutdown_connections 2023-10-06T11:22:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.835+0000 7fa37f2ae700 1 -- 172.21.15.7:0/3652074585 wait complete. 2023-10-06T11:22:40.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 Processor -- start 2023-10-06T11:22:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 -- start start 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74bc0cefe0 0x7f74bc06d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f74bc0ccdc0 0x7f74bc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc06e770 con 0x7f74bc159eb0 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0ceb50 con 0x7f74bc0cefe0 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74c116f700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0ce220 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74ba59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74bc0cefe0 0x7f74bc06d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74ba59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74bc0cefe0 0x7f74bc06d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44678/0 (socket says 172.21.15.7:44678) 2023-10-06T11:22:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74ba59c700 1 -- 172.21.15.7:0/1569429659 learned_addr learned my addr 172.21.15.7:0/1569429659 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.963+0000 7f74bb59e700 1 --2- 172.21.15.7:0/1569429659 >> v2:172.21.15.50:3300/0 conn(0x7f74bc0ccdc0 0x7f74bc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74ba59c700 1 -- 172.21.15.7:0/1569429659 >> v2:172.21.15.50:3300/0 conn(0x7f74bc0ccdc0 msgr2=0x7f74bc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74ba59c700 1 --2- 172.21.15.7:0/1569429659 >> v2:172.21.15.50:3300/0 conn(0x7f74bc0ccdc0 0x7f74bc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74ba59c700 1 -- 172.21.15.7:0/1569429659 >> v1:172.21.15.50:6789/0 conn(0x7f74bc159eb0 legacy=0x7f74bc0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74ba59c700 1 -- 172.21.15.7:0/1569429659 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0ce500 con 0x7f74bc0cefe0 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74ba59c700 1 --2- 172.21.15.7:0/1569429659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cefe0 0x7f74bc06d9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0c37f0 tx=0x7f74ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e37eb84b9c6467 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74b9d9b700 1 -- 172.21.15.7:0/1569429659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009e70 con 0x7f74bc0cefe0 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74b9d9b700 1 -- 172.21.15.7:0/1569429659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00b860 con 0x7f74bc0cefe0 2023-10-06T11:22:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74b9d9b700 1 -- 172.21.15.7:0/1569429659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00bc00 con 0x7f74bc0cefe0 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 -- 172.21.15.7:0/1569429659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cefe0 msgr2=0x7f74bc06d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 --2- 172.21.15.7:0/1569429659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cefe0 0x7f74bc06d9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0c37f0 tx=0x7f74ac009250 comp rx=0 tx=0).stop 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 -- 172.21.15.7:0/1569429659 shutdown_connections 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 --2- 172.21.15.7:0/1569429659 >> v2:172.21.15.50:3300/0 conn(0x7f74bc0ccdc0 0x7f74bc06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 --2- 172.21.15.7:0/1569429659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cefe0 0x7f74bc06d9d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 -- 172.21.15.7:0/1569429659 >> 172.21.15.7:0/1569429659 conn(0x7f74bc067e30 msgr2=0x7f74bc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 -- 172.21.15.7:0/1569429659 shutdown_connections 2023-10-06T11:22:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.967+0000 7f74c116f700 1 -- 172.21.15.7:0/1569429659 wait complete. 2023-10-06T11:22:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 Processor -- start 2023-10-06T11:22:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- start start 2023-10-06T11:22:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f74bc0cefe0 0x7f74bc169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74bc159eb0 0x7f74bc069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0b50e0 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc16a270 con 0x7f74bc159eb0 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc16af00 con 0x7f74bc0cefe0 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44692/0 (socket says 172.21.15.7:44692) 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 -- 172.21.15.7:0/1899063091 learned_addr learned my addr 172.21.15.7:0/1899063091 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74ba59c700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f74bc0cefe0 0x7f74bc169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bb59e700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74bc159eb0 0x7f74bc069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 -- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f74bc0cefe0 msgr2=0x7f74bc169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f74bc0cefe0 0x7f74bc169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 -- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74bc159eb0 msgr2=0x7f74bc069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74bc159eb0 0x7f74bc069c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 -- 172.21.15.7:0/1899063091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0094f0 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74bad9d700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c4e0 tx=0x7f74b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0011590 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b0011710 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc16a610 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0011ad0 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.971+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc16ab90 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.975+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc04be00 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.975+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f74b001b3b0 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.975+0000 7f74a2ffd700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 0x7f74a8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.975+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f74b007bff0 con 0x7f74bc0ccdc0 2023-10-06T11:22:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.975+0000 7f74ba59c700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 0x7f74a8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:40.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.979+0000 7f74ba59c700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 0x7f74a8062a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0cd780 tx=0x7f74ac017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:40.983+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74b004c150 con 0x7f74bc0ccdc0 2023-10-06T11:22:41.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74bc0cdd30 con 0x7f74bc0ccdc0 2023-10-06T11:22:41.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74a2ffd700 1 -- 172.21.15.7:0/1899063091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74b004f770 con 0x7f74bc0ccdc0 2023-10-06T11:22:41.151 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 msgr2=0x7f74a8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 0x7f74a8062a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0cd780 tx=0x7f74ac017000 comp rx=0 tx=0).stop 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 msgr2=0x7f74bc0b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c4e0 tx=0x7f74b000c7f0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 shutdown_connections 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f74a8060580 0x7f74a8062a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74bc159eb0 0x7f74bc069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f74bc0cefe0 0x7f74bc169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 --2- 172.21.15.7:0/1899063091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0ccdc0 0x7f74bc0b49e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.147+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 >> 172.21.15.7:0/1899063091 conn(0x7f74bc067e30 msgr2=0x7f74bc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.151+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 shutdown_connections 2023-10-06T11:22:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.151+0000 7f74c116f700 1 -- 172.21.15.7:0/1899063091 wait complete. 2023-10-06T11:22:41.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 Processor -- start 2023-10-06T11:22:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 -- start start 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d0c159200 0x7f9d0c068f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c06a4c0 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0ae840 con 0x7f9d0c159200 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d1189b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0698b0 con 0x7f9d0c06a950 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d0c159200 0x7f9d0c068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9d0c0c82d0 0x7f9d0c0ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0b7fe700 1 -- 172.21.15.7:0/3516682445 learned_addr learned my addr 172.21.15.7:0/3516682445 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0a7fc700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d09ffb700 1 -- 172.21.15.7:0/3516682445 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278489006 0 0) 0x7f9d0c06a4c0 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d09ffb700 1 -- 172.21.15.7:0/3516682445 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf8003570 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 -- 172.21.15.7:0/3516682445 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 msgr2=0x7f9d0c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 -- 172.21.15.7:0/3516682445 >> v1:172.21.15.7:6789/0 conn(0x7f9d0c0c82d0 legacy=0x7f9d0c0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 -- 172.21.15.7:0/3516682445 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c069b90 con 0x7f9d0c159200 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.279+0000 7f9d0affd700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c159200 0x7f9d0c068f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098e0 tx=0x7f9cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860c7ae3a280454e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d09ffb700 1 -- 172.21.15.7:0/3516682445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011630 con 0x7f9d0c159200 2023-10-06T11:22:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d09ffb700 1 -- 172.21.15.7:0/3516682445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc011790 con 0x7f9d0c159200 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d09ffb700 1 -- 172.21.15.7:0/3516682445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011b30 con 0x7f9d0c159200 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- 172.21.15.7:0/3516682445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c159200 msgr2=0x7f9d0c068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c159200 0x7f9d0c068f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098e0 tx=0x7f9cfc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- 172.21.15.7:0/3516682445 shutdown_connections 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- 172.21.15.7:0/3516682445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c159200 0x7f9d0c068f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- 172.21.15.7:0/3516682445 >> 172.21.15.7:0/3516682445 conn(0x7f9d0c066840 msgr2=0x7f9d0c066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- 172.21.15.7:0/3516682445 shutdown_connections 2023-10-06T11:22:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- 172.21.15.7:0/3516682445 wait complete. 2023-10-06T11:22:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 Processor -- start 2023-10-06T11:22:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- start start 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d0c159200 0x7f9d0c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c5430 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c16cca0 con 0x7f9d0c159200 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.283+0000 7f9d1189b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c16cf80 con 0x7f9d0c06a950 2023-10-06T11:22:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44722/0 (socket says 172.21.15.7:44722) 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 -- 172.21.15.7:0/1031266786 learned_addr learned my addr 172.21.15.7:0/1031266786 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0affd700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0b7fe700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d0c159200 0x7f9d0c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 -- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 msgr2=0x7f9d0c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 -- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d0c159200 msgr2=0x7f9d0c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d0c159200 0x7f9d0c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 -- 172.21.15.7:0/1031266786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009420 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d0a7fc700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400ec80 tx=0x7f9cf4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400c9d0 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf400cb30 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c16d260 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4013470 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0b35a0 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.287+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c04be00 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.291+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9cf4013860 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.291+0000 7f9d02ffd700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 0x7f9cec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.291+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9cf407d800 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.291+0000 7f9d0affd700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 0x7f9cec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.291+0000 7f9d0affd700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 0x7f9cec062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0091e0 tx=0x7f9cfc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.299+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9cf404d980 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d0c069e70 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d02ffd700 1 -- 172.21.15.7:0/1031266786 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cf4050fa0 con 0x7f9d0c0c82d0 2023-10-06T11:22:41.468 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:41.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 msgr2=0x7f9cec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 0x7f9cec062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0091e0 tx=0x7f9cfc017000 comp rx=0 tx=0).stop 2023-10-06T11:22:41.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 msgr2=0x7f9d0c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.463+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400ec80 tx=0x7f9cf4009120 comp rx=0 tx=0).stop 2023-10-06T11:22:41.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 shutdown_connections 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9cec060550 0x7f9cec062a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d0c159200 0x7f9d0c156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0c0c82d0 0x7f9d0c155fe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 --2- 172.21.15.7:0/1031266786 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9d0c06a950 0x7f9d0c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 >> 172.21.15.7:0/1031266786 conn(0x7f9d0c066840 msgr2=0x7f9d0c066c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 shutdown_connections 2023-10-06T11:22:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.467+0000 7f9d1189b700 1 -- 172.21.15.7:0/1031266786 wait complete. 2023-10-06T11:22:41.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:41.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 Processor -- start 2023-10-06T11:22:41.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 -- start start 2023-10-06T11:22:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3406e770 con 0x7f6d3406e0e0 2023-10-06T11:22:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d340ceb50 con 0x7f6d340cefe0 2023-10-06T11:22:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.583+0000 7f6d39c4d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3406e520 con 0x7f6d34159eb0 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d33fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6d3406e0e0 0x7f6d340ca390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35790/0 (socket says 172.21.15.7:35790) 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d33fff700 1 -- 172.21.15.7:0/2094937062 learned_addr learned my addr 172.21.15.7:0/2094937062 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d337fe700 1 --2- 172.21.15.7:0/2094937062 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445481999 0 0) 0x7f6d3406e770 con 0x7f6d3406e0e0 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d20003570 con 0x7f6d3406e0e0 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646322858 0 0) 0x7f6d340ceb50 con 0x7f6d340cefe0 2023-10-06T11:22:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d3406e770 con 0x7f6d340cefe0 2023-10-06T11:22:41.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d337fe700 1 -- 172.21.15.7:0/2094937062 >> v1:172.21.15.50:6789/0 conn(0x7f6d340cefe0 legacy=0x7f6d3406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d337fe700 1 -- 172.21.15.7:0/2094937062 >> v1:172.21.15.7:6789/0 conn(0x7f6d3406e0e0 legacy=0x7f6d340ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d337fe700 1 -- 172.21.15.7:0/2094937062 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d340ce220 con 0x7f6d34159eb0 2023-10-06T11:22:41.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d337fe700 1 --2- 172.21.15.7:0/2094937062 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009910 tx=0x7f6d1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8004da13800742 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c011630 con 0x7f6d34159eb0 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c011790 con 0x7f6d34159eb0 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d327fc700 1 -- 172.21.15.7:0/2094937062 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c011b10 con 0x7f6d34159eb0 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 -- 172.21.15.7:0/2094937062 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 msgr2=0x7f6d340cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/2094937062 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009910 tx=0x7f6d1c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 -- 172.21.15.7:0/2094937062 shutdown_connections 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/2094937062 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc6b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 -- 172.21.15.7:0/2094937062 >> 172.21.15.7:0/2094937062 conn(0x7f6d34067e30 msgr2=0x7f6d34068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 -- 172.21.15.7:0/2094937062 shutdown_connections 2023-10-06T11:22:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.587+0000 7f6d39c4d700 1 -- 172.21.15.7:0/2094937062 wait complete. 2023-10-06T11:22:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 Processor -- start 2023-10-06T11:22:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- start start 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d3406e0e0 0x7f6d34158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c8c50 con 0x7f6d340cefe0 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c8f30 con 0x7f6d3406e0e0 2023-10-06T11:22:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c9210 con 0x7f6d34159eb0 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44734/0 (socket says 172.21.15.7:44734) 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 -- 172.21.15.7:0/953504932 learned_addr learned my addr 172.21.15.7:0/953504932 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d337fe700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d3406e0e0 0x7f6d34158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d33fff700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 -- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 msgr2=0x7f6d340cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 -- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d3406e0e0 msgr2=0x7f6d34158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d3406e0e0 0x7f6d34158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 -- 172.21.15.7:0/953504932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c009420 con 0x7f6d340cefe0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d32ffd700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c7c0 tx=0x7f6d24007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24010510 con 0x7f6d340cefe0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d24002e90 con 0x7f6d340cefe0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d340c94f0 con 0x7f6d340cefe0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240108e0 con 0x7f6d340cefe0 2023-10-06T11:22:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.591+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d340c86f0 con 0x7f6d340cefe0 2023-10-06T11:22:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6d24010cd0 con 0x7f6d340cefe0 2023-10-06T11:22:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3404be00 con 0x7f6d340cefe0 2023-10-06T11:22:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d2b7fe700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 0x7f6d14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6d2407c3e0 con 0x7f6d340cefe0 2023-10-06T11:22:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d337fe700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 0x7f6d14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.595+0000 7f6d337fe700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 0x7f6d14062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009860 tx=0x7f6d1c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.603+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6d2404c4e0 con 0x7f6d340cefe0 2023-10-06T11:22:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.771+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6d340cdd30 con 0x7f6d340cefe0 2023-10-06T11:22:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d2b7fe700 1 -- 172.21.15.7:0/953504932 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6d240170a0 con 0x7f6d340cefe0 2023-10-06T11:22:41.778 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 msgr2=0x7f6d14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 0x7f6d14062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009860 tx=0x7f6d1c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 msgr2=0x7f6d340c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c7c0 tx=0x7f6d24007de0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 shutdown_connections 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6d14060550 0x7f6d14062a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6d34159eb0 0x7f6d340cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d340cefe0 0x7f6d340c81b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 --2- 172.21.15.7:0/953504932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d3406e0e0 0x7f6d34158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 >> 172.21.15.7:0/953504932 conn(0x7f6d34067e30 msgr2=0x7f6d34069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 shutdown_connections 2023-10-06T11:22:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.775+0000 7f6d39c4d700 1 -- 172.21.15.7:0/953504932 wait complete. 2023-10-06T11:22:41.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.899+0000 7ff159def700 1 Processor -- start 2023-10-06T11:22:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.899+0000 7ff159def700 1 -- start start 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff1540cbf30 0x7ff1540cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff154069d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1540c8cf0 con 0x7ff1540ca230 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1540c8400 con 0x7ff1540cbf30 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540c86a0 con 0x7ff1540ca6b0 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff14bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff1540ca230 0x7ff154069630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35800/0 (socket says 172.21.15.7:35800) 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff14bfff700 1 -- 172.21.15.7:0/2975283157 learned_addr learned my addr 172.21.15.7:0/2975283157 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 --2- 172.21.15.7:0/2975283157 >> v2:172.21.15.50:3300/0 conn(0x7ff1540cbf30 0x7ff1540cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff153fff700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff154069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351355602 0 0) 0x7ff1540c8cf0 con 0x7ff1540ca230 2023-10-06T11:22:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff140003570 con 0x7ff1540ca230 2023-10-06T11:22:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 -- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 msgr2=0x7ff154069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff154069d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218528314 0 0) 0x7ff140003570 con 0x7ff1540ca230 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 -- 172.21.15.7:0/2975283157 >> v1:172.21.15.7:6789/0 conn(0x7ff1540ca230 legacy=0x7ff154069630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 -- 172.21.15.7:0/2975283157 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540c8980 con 0x7ff1540cbf30 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff1537fe700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff1540cc300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1440098e0 tx=0x7ff1440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55296cbd509fc53b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c001470 con 0x7ff1540cbf30 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144011420 con 0x7ff1540cbf30 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff152ffd700 1 -- 172.21.15.7:0/2975283157 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1440117c0 con 0x7ff1540cbf30 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 msgr2=0x7ff1540cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff1540cc300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1440098e0 tx=0x7ff1440091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- 172.21.15.7:0/2975283157 shutdown_connections 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff154069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 --2- 172.21.15.7:0/2975283157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff1540cc300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- 172.21.15.7:0/2975283157 >> 172.21.15.7:0/2975283157 conn(0x7ff154066840 msgr2=0x7ff154066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- 172.21.15.7:0/2975283157 shutdown_connections 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.903+0000 7ff159def700 1 -- 172.21.15.7:0/2975283157 wait complete. 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 Processor -- start 2023-10-06T11:22:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 -- start start 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff15413b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff154159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154157520 con 0x7ff1540ca230 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154168e00 con 0x7ff1540cbf30 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff159def700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15415a280 con 0x7ff1540ca6b0 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff1537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff1537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44750/0 (socket says 172.21.15.7:44750) 2023-10-06T11:22:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.907+0000 7ff1537fe700 1 -- 172.21.15.7:0/4198824428 learned_addr learned my addr 172.21.15.7:0/4198824428 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff14bfff700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff15413b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff153fff700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff154159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 -- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 msgr2=0x7ff15413b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff15413b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 -- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 msgr2=0x7ff154159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff154159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 -- 172.21.15.7:0/4198824428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144009420 con 0x7ff1540ca230 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff153fff700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff154159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff1537fe700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1440030c0 tx=0x7ff144016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14401cbe0 con 0x7ff1540ca230 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14401cd40 con 0x7ff1540ca230 2023-10-06T11:22:41.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff154168210 con 0x7ff1540ca230 2023-10-06T11:22:41.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14401a6a0 con 0x7ff1540ca230 2023-10-06T11:22:41.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1541687b0 con 0x7ff1540ca230 2023-10-06T11:22:41.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.911+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15404be00 con 0x7ff1540ca230 2023-10-06T11:22:41.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.915+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff14401aad0 con 0x7ff1540ca230 2023-10-06T11:22:41.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.915+0000 7ff14b7fe700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 0x7ff134062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:41.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.915+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff14407e4b0 con 0x7ff1540ca230 2023-10-06T11:22:41.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.915+0000 7ff14bfff700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 0x7ff134062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:41.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.915+0000 7ff14bfff700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 0x7ff134062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff1540c9fa0 tx=0x7ff13c001aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:41.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:41.923+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff14404e650 con 0x7ff1540ca230 2023-10-06T11:22:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.083+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff154159b20 con 0x7ff1540ca230 2023-10-06T11:22:42.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.087+0000 7ff14b7fe700 1 -- 172.21.15.7:0/4198824428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7ff144051c70 con 0x7ff1540ca230 2023-10-06T11:22:42.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:42.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:42.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 msgr2=0x7ff134062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 0x7ff134062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff1540c9fa0 tx=0x7ff13c001aa0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 msgr2=0x7ff154167c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1440030c0 tx=0x7ff144016000 comp rx=0 tx=0).stop 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 shutdown_connections 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff134060550 0x7ff134062a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1540cbf30 0x7ff154159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff1540ca6b0 0x7ff15413b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 --2- 172.21.15.7:0/4198824428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1540ca230 0x7ff154167c20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 >> 172.21.15.7:0/4198824428 conn(0x7ff154066840 msgr2=0x7ff1541524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 shutdown_connections 2023-10-06T11:22:42.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.091+0000 7ff159def700 1 -- 172.21.15.7:0/4198824428 wait complete. 2023-10-06T11:22:42.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:42.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:42.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 Processor -- start 2023-10-06T11:22:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 -- start start 2023-10-06T11:22:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f57b806a3e0 0x7f57b81601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80c7af0 con 0x7f57b81554c0 2023-10-06T11:22:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80cb3e0 con 0x7f57b81550f0 2023-10-06T11:22:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57be5e8700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80cb6c0 con 0x7f57b806a3e0 2023-10-06T11:22:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57bcde5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57b81554c0 0x7f57b8069cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51994/0 (socket says 172.21.15.7:51994) 2023-10-06T11:22:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57bcde5700 1 -- 172.21.15.7:0/3683295755 learned_addr learned my addr 172.21.15.7:0/3683295755 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.219+0000 7f57bdde7700 1 --2- 172.21.15.7:0/3683295755 >> v2:172.21.15.50:3300/0 conn(0x7f57b806a3e0 0x7f57b81601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800996508 0 0) 0x7f57b80c7af0 con 0x7f57b81554c0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a4003570 con 0x7f57b81554c0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793240739 0 0) 0x7f57b80cb3e0 con 0x7f57b81550f0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b80c7af0 con 0x7f57b81550f0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946866687 0 0) 0x7f57a4003570 con 0x7f57b81554c0 2023-10-06T11:22:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57bdde7700 1 -- 172.21.15.7:0/3683295755 >> v1:172.21.15.50:6789/0 conn(0x7f57b81550f0 legacy=0x7f57b80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57bdde7700 1 -- 172.21.15.7:0/3683295755 >> v1:172.21.15.7:6789/0 conn(0x7f57b81554c0 legacy=0x7f57b8069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57bdde7700 1 -- 172.21.15.7:0/3683295755 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80c7780 con 0x7f57b806a3e0 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57bdde7700 1 --2- 172.21.15.7:0/3683295755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b806a3e0 0x7f57b81601d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57b000af70 tx=0x7f57b000a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ec0286bd852bae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000f070 con 0x7f57b806a3e0 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000a0d0 con 0x7f57b806a3e0 2023-10-06T11:22:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57affff700 1 -- 172.21.15.7:0/3683295755 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4001470 con 0x7f57b806a3e0 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 -- 172.21.15.7:0/3683295755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b806a3e0 msgr2=0x7f57b81601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 --2- 172.21.15.7:0/3683295755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b806a3e0 0x7f57b81601d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57b000af70 tx=0x7f57b000a890 comp rx=0 tx=0).stop 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 -- 172.21.15.7:0/3683295755 shutdown_connections 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 --2- 172.21.15.7:0/3683295755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b806a3e0 0x7f57b81601d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 -- 172.21.15.7:0/3683295755 >> 172.21.15.7:0/3683295755 conn(0x7f57b8067e30 msgr2=0x7f57b8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 -- 172.21.15.7:0/3683295755 shutdown_connections 2023-10-06T11:22:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.223+0000 7f57be5e8700 1 -- 172.21.15.7:0/3683295755 wait complete. 2023-10-06T11:22:42.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 Processor -- start 2023-10-06T11:22:42.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 -- start start 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b81550f0 0x7f57b80c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f57b80c5aa0 0x7f57b81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8165950 con 0x7f57b806a3e0 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8164d40 con 0x7f57b81550f0 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8165020 con 0x7f57b80c5aa0 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47548/0 (socket says 172.21.15.7:47548) 2023-10-06T11:22:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 -- 172.21.15.7:0/874317727 learned_addr learned my addr 172.21.15.7:0/874317727 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bcde5700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b81550f0 0x7f57b80c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bdde7700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f57b80c5aa0 0x7f57b81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 -- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f57b80c5aa0 msgr2=0x7f57b81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f57b80c5aa0 0x7f57b81540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 -- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b81550f0 msgr2=0x7f57b80c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b81550f0 0x7f57b80c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 -- 172.21.15.7:0/874317727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b000ab30 con 0x7f57b806a3e0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57bd5e6700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c790 tx=0x7f57a8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010510 con 0x7f57b806a3e0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8010670 con 0x7f57b806a3e0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010a10 con 0x7f57b806a3e0 2023-10-06T11:22:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.227+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b8165300 con 0x7f57b806a3e0 2023-10-06T11:22:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b81649d0 con 0x7f57b806a3e0 2023-10-06T11:22:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b806cf60 con 0x7f57b806a3e0 2023-10-06T11:22:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f57a801b3b0 con 0x7f57b806a3e0 2023-10-06T11:22:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57ad7fa700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 0x7f579c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f57a807c4c0 con 0x7f57b806a3e0 2023-10-06T11:22:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.231+0000 7f57bcde5700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 0x7f579c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.235+0000 7f57bcde5700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 0x7f579c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f57b8158d60 tx=0x7f57b4001950 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.243+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f57a804c660 con 0x7f57b806a3e0 2023-10-06T11:22:42.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.399+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f57b806aa40 con 0x7f57b806a3e0 2023-10-06T11:22:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57ad7fa700 1 -- 172.21.15.7:0/874317727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f57a80170a0 con 0x7f57b806a3e0 2023-10-06T11:22:42.406 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 msgr2=0x7f579c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 0x7f579c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f57b8158d60 tx=0x7f57b4001950 comp rx=0 tx=0).stop 2023-10-06T11:22:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 msgr2=0x7f57b8156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c790 tx=0x7f57a8007e10 comp rx=0 tx=0).stop 2023-10-06T11:22:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 shutdown_connections 2023-10-06T11:22:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f579c060550 0x7f579c062a00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f57b80c5aa0 0x7f57b81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57b81550f0 0x7f57b80c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 --2- 172.21.15.7:0/874317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b806a3e0 0x7f57b8156c30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 >> 172.21.15.7:0/874317727 conn(0x7f57b8067e30 msgr2=0x7f57b80c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 shutdown_connections 2023-10-06T11:22:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.403+0000 7f57be5e8700 1 -- 172.21.15.7:0/874317727 wait complete. 2023-10-06T11:22:42.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:42.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 Processor -- start 2023-10-06T11:22:42.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 -- start start 2023-10-06T11:22:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80ac400 0x7effe80ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7effe80b00f0 0x7effe80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe80ad4a0 con 0x7effe80aab90 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80af7a0 con 0x7effe80ac400 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effef383700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe8155fb0 con 0x7effe80b00f0 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80ac400 0x7effe80ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80ac400 0x7effe80ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47564/0 (socket says 172.21.15.7:47564) 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 -- 172.21.15.7:0/3942692979 learned_addr learned my addr 172.21.15.7:0/3942692979 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effeeb82700 1 --2- 172.21.15.7:0/3942692979 >> v2:172.21.15.50:3300/0 conn(0x7effe80b00f0 0x7effe80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 -- 172.21.15.7:0/3942692979 >> v2:172.21.15.50:3300/0 conn(0x7effe80b00f0 msgr2=0x7effe80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 --2- 172.21.15.7:0/3942692979 >> v2:172.21.15.50:3300/0 conn(0x7effe80b00f0 0x7effe80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 -- 172.21.15.7:0/3942692979 >> v1:172.21.15.50:6789/0 conn(0x7effe80aab90 legacy=0x7effe80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 -- 172.21.15.7:0/3942692979 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80b0a80 con 0x7effe80ac400 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.527+0000 7effedb80700 1 --2- 172.21.15.7:0/3942692979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80ac400 0x7effe80ad930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7effe8068a80 tx=0x7effd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a2d1e7907cdd96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effed37f700 1 -- 172.21.15.7:0/3942692979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8009ea0 con 0x7effe80ac400 2023-10-06T11:22:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effed37f700 1 -- 172.21.15.7:0/3942692979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd800b820 con 0x7effe80ac400 2023-10-06T11:22:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effed37f700 1 -- 172.21.15.7:0/3942692979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800bbc0 con 0x7effe80ac400 2023-10-06T11:22:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 -- 172.21.15.7:0/3942692979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80ac400 msgr2=0x7effe80ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 --2- 172.21.15.7:0/3942692979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80ac400 0x7effe80ad930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7effe8068a80 tx=0x7effd8009250 comp rx=0 tx=0).stop 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 -- 172.21.15.7:0/3942692979 shutdown_connections 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 --2- 172.21.15.7:0/3942692979 >> v2:172.21.15.50:3300/0 conn(0x7effe80b00f0 0x7effe80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 --2- 172.21.15.7:0/3942692979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80ac400 0x7effe80ad930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 -- 172.21.15.7:0/3942692979 >> 172.21.15.7:0/3942692979 conn(0x7effe80cc3c0 msgr2=0x7effe8067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 -- 172.21.15.7:0/3942692979 shutdown_connections 2023-10-06T11:22:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.531+0000 7effef383700 1 -- 172.21.15.7:0/3942692979 wait complete. 2023-10-06T11:22:42.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 Processor -- start 2023-10-06T11:22:42.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 -- start start 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7effe80aab90 0x7effe806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effe80ac400 0x7effe8140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8157560 con 0x7effe80b00f0 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8157840 con 0x7effe80ac400 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effef383700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe8157b20 con 0x7effe80aab90 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47576/0 (socket says 172.21.15.7:47576) 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 -- 172.21.15.7:0/814563189 learned_addr learned my addr 172.21.15.7:0/814563189 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effee381700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7effe80aab90 0x7effe806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effedb80700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effe80ac400 0x7effe8140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 -- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7effe80aab90 msgr2=0x7effe806a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7effe80aab90 0x7effe806a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 -- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effe80ac400 msgr2=0x7effe8140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effe80ac400 0x7effe8140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.535+0000 7effeeb82700 1 -- 172.21.15.7:0/814563189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80094f0 con 0x7effe80b00f0 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effeeb82700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effe0009df0 tx=0x7effe000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe000caa0 con 0x7effe80b00f0 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe000f040 con 0x7effe80b00f0 2023-10-06T11:22:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effef383700 1 -- 172.21.15.7:0/814563189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe8157e00 con 0x7effe80b00f0 2023-10-06T11:22:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe000ce70 con 0x7effe80b00f0 2023-10-06T11:22:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effef383700 1 -- 172.21.15.7:0/814563189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe806a890 con 0x7effe80b00f0 2023-10-06T11:22:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effef383700 1 -- 172.21.15.7:0/814563189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8158300 con 0x7effe80b00f0 2023-10-06T11:22:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7effe0014690 con 0x7effe80b00f0 2023-10-06T11:22:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 0x7effcc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7effe007d3f0 con 0x7effe80b00f0 2023-10-06T11:22:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.539+0000 7effee381700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 0x7effcc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.543+0000 7effee381700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 0x7effcc062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effe40039f0 tx=0x7effe4007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.551+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7effe0051020 con 0x7effe80b00f0 2023-10-06T11:22:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 -- 172.21.15.7:0/814563189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effe80ac890 con 0x7effe80b00f0 2023-10-06T11:22:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effde7fc700 1 -- 172.21.15.7:0/814563189 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7effe0050bb0 con 0x7effe80b00f0 2023-10-06T11:22:42.719 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 -- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 msgr2=0x7effcc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 0x7effcc062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effe40039f0 tx=0x7effe4007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 -- 172.21.15.7:0/814563189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 msgr2=0x7effe81414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effe0009df0 tx=0x7effe000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 -- 172.21.15.7:0/814563189 shutdown_connections 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7effcc060550 0x7effcc062a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b00f0 0x7effe81414a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effe80ac400 0x7effe8140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 --2- 172.21.15.7:0/814563189 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7effe80aab90 0x7effe806a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.715+0000 7effef383700 1 -- 172.21.15.7:0/814563189 >> 172.21.15.7:0/814563189 conn(0x7effe80cc3c0 msgr2=0x7effe8154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.719+0000 7effef383700 1 -- 172.21.15.7:0/814563189 shutdown_connections 2023-10-06T11:22:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.719+0000 7effef383700 1 -- 172.21.15.7:0/814563189 wait complete. 2023-10-06T11:22:42.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 Processor -- start 2023-10-06T11:22:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 -- start start 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 0x7fc5540b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5540b41e0 0x7fc55406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc55406dc40 0x7fc554159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540b0f80 con 0x7fc5540b41e0 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55406e0c0 con 0x7fc5540b3e10 2023-10-06T11:22:42.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55359e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b1190 con 0x7fc55406dc40 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5540b41e0 0x7fc55406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5540b41e0 0x7fc55406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47590/0 (socket says 172.21.15.7:47590) 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 -- 172.21.15.7:0/3968339597 learned_addr learned my addr 172.21.15.7:0/3968339597 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc552d9d700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc55406dc40 0x7fc554159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55259c700 1 --2- 172.21.15.7:0/3968339597 >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 0x7fc5540b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 -- 172.21.15.7:0/3968339597 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc55406dc40 msgr2=0x7fc554159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc55406dc40 0x7fc554159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 -- 172.21.15.7:0/3968339597 >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 msgr2=0x7fc5540b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 --2- 172.21.15.7:0/3968339597 >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 0x7fc5540b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc551d9b700 1 -- 172.21.15.7:0/3968339597 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540b1470 con 0x7fc5540b41e0 2023-10-06T11:22:42.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.843+0000 7fc55259c700 1 --2- 172.21.15.7:0/3968339597 >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 0x7fc5540b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:42.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc551d9b700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5540b41e0 0x7fc55406d700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc55406b590 tx=0x7fc544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609fb42127589fe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55159a700 1 -- 172.21.15.7:0/3968339597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544009e70 con 0x7fc5540b41e0 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55159a700 1 -- 172.21.15.7:0/3968339597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400b860 con 0x7fc5540b41e0 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55159a700 1 -- 172.21.15.7:0/3968339597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400bc00 con 0x7fc5540b41e0 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 -- 172.21.15.7:0/3968339597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5540b41e0 msgr2=0x7fc55406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5540b41e0 0x7fc55406d700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc55406b590 tx=0x7fc544009250 comp rx=0 tx=0).stop 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 -- 172.21.15.7:0/3968339597 shutdown_connections 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc55406dc40 0x7fc554159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 --2- 172.21.15.7:0/3968339597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5540b41e0 0x7fc55406d700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 --2- 172.21.15.7:0/3968339597 >> v2:172.21.15.50:3300/0 conn(0x7fc5540b3e10 0x7fc5540b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 -- 172.21.15.7:0/3968339597 >> 172.21.15.7:0/3968339597 conn(0x7fc5540cfd00 msgr2=0x7fc5540d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 -- 172.21.15.7:0/3968339597 shutdown_connections 2023-10-06T11:22:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.847+0000 7fc55359e700 1 -- 172.21.15.7:0/3968339597 wait complete. 2023-10-06T11:22:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 Processor -- start 2023-10-06T11:22:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- start start 2023-10-06T11:22:42.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5540b3e10 0x7fc554155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b41e0 0x7fc554156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55416cf70 con 0x7fc55406dc40 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55416daf0 con 0x7fc5540b41e0 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55416d1c0 con 0x7fc5540b3e10 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47606/0 (socket says 172.21.15.7:47606) 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 -- 172.21.15.7:0/1081638874 learned_addr learned my addr 172.21.15.7:0/1081638874 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc552d9d700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b41e0 0x7fc554156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc551d9b700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5540b3e10 0x7fc554155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 -- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5540b3e10 msgr2=0x7fc554155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5540b3e10 0x7fc554155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 -- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b41e0 msgr2=0x7fc554156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b41e0 0x7fc554156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 -- 172.21.15.7:0/1081638874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440094f0 con 0x7fc55406dc40 2023-10-06T11:22:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55259c700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e900 tx=0x7fc548008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548013410 con 0x7fc55406dc40 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548013570 con 0x7fc55406dc40 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55416d4a0 con 0x7fc55406dc40 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548013910 con 0x7fc55406dc40 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.851+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55416a7c0 con 0x7fc55406dc40 2023-10-06T11:22:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.855+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55406c940 con 0x7fc55406dc40 2023-10-06T11:22:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.855+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc548013cc0 con 0x7fc55406dc40 2023-10-06T11:22:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.855+0000 7fc5427fc700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 0x7fc534062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.855+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc54807c9a0 con 0x7fc55406dc40 2023-10-06T11:22:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.855+0000 7fc551d9b700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 0x7fc534062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.859+0000 7fc551d9b700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 0x7fc534062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc55406b590 tx=0x7fc544017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:42.867+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc54804caa0 con 0x7fc55406dc40 2023-10-06T11:22:43.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.027+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5540b1750 con 0x7fc55406dc40 2023-10-06T11:22:43.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.027+0000 7fc5427fc700 1 -- 172.21.15.7:0/1081638874 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5480500c0 con 0x7fc55406dc40 2023-10-06T11:22:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:43.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 msgr2=0x7fc534062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 0x7fc534062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc55406b590 tx=0x7fc544017000 comp rx=0 tx=0).stop 2023-10-06T11:22:43.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 msgr2=0x7fc55416a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e900 tx=0x7fc548008040 comp rx=0 tx=0).stop 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 shutdown_connections 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc534060550 0x7fc534062a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b41e0 0x7fc554156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5540b3e10 0x7fc554155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 --2- 172.21.15.7:0/1081638874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55406dc40 0x7fc55416a280 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 >> 172.21.15.7:0/1081638874 conn(0x7fc5540cfd00 msgr2=0x7fc554069460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 shutdown_connections 2023-10-06T11:22:43.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.031+0000 7fc55359e700 1 -- 172.21.15.7:0/1081638874 wait complete. 2023-10-06T11:22:43.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:43.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.159+0000 7fb8f759e700 1 Processor -- start 2023-10-06T11:22:43.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.159+0000 7fb8f759e700 1 -- start start 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb8f80b21d0 0x7fb8f80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8f80ccdb0 0x7fb8f80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f815d320 con 0x7fb8f8159f20 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f815a390 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f80ce940 con 0x7fb8f80b21d0 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f5d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8f8159f20 0x7fb8f80cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52004/0 (socket says 172.21.15.7:52004) 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8f80ccdb0 0x7fb8f80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f5d9b700 1 -- 172.21.15.7:0/40583674 learned_addr learned my addr 172.21.15.7:0/40583674 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 --2- 172.21.15.7:0/40583674 >> v2:172.21.15.50:3300/0 conn(0x7fb8f80b21d0 0x7fb8f80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f559a700 1 -- 172.21.15.7:0/40583674 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959735275 0 0) 0x7fb8f815d320 con 0x7fb8f8159f20 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 -- 172.21.15.7:0/40583674 >> v2:172.21.15.7:3300/0 conn(0x7fb8f80ccdb0 msgr2=0x7fb8f80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 --2- 172.21.15.7:0/40583674 >> v2:172.21.15.7:3300/0 conn(0x7fb8f80ccdb0 0x7fb8f80ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 -- 172.21.15.7:0/40583674 >> v1:172.21.15.7:6789/0 conn(0x7fb8f8159f20 legacy=0x7fb8f80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 -- 172.21.15.7:0/40583674 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f815c9f0 con 0x7fb8f80b21d0 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f659c700 1 --2- 172.21.15.7:0/40583674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f80b21d0 0x7fb8f80ce260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80cfc00 tx=0x7fb8e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d467d26706b0f123 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f559a700 1 -- 172.21.15.7:0/40583674 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011630 con 0x7fb8f80b21d0 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f559a700 1 -- 172.21.15.7:0/40583674 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8011790 con 0x7fb8f80b21d0 2023-10-06T11:22:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f559a700 1 -- 172.21.15.7:0/40583674 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011b30 con 0x7fb8f80b21d0 2023-10-06T11:22:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- 172.21.15.7:0/40583674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f80b21d0 msgr2=0x7fb8f80ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 --2- 172.21.15.7:0/40583674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f80b21d0 0x7fb8f80ce260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80cfc00 tx=0x7fb8e80091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- 172.21.15.7:0/40583674 shutdown_connections 2023-10-06T11:22:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 --2- 172.21.15.7:0/40583674 >> v2:172.21.15.7:3300/0 conn(0x7fb8f80ccdb0 0x7fb8f80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 --2- 172.21.15.7:0/40583674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f80b21d0 0x7fb8f80ce260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- 172.21.15.7:0/40583674 >> 172.21.15.7:0/40583674 conn(0x7fb8f8067e30 msgr2=0x7fb8f8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- 172.21.15.7:0/40583674 shutdown_connections 2023-10-06T11:22:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.163+0000 7fb8f759e700 1 -- 172.21.15.7:0/40583674 wait complete. 2023-10-06T11:22:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 Processor -- start 2023-10-06T11:22:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 -- start start 2023-10-06T11:22:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8f80b21d0 0x7fb8f80c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f8159f20 0x7fb8f815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80c6d50 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80c75f0 con 0x7fb8f8159f20 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80c83b0 con 0x7fb8f80b21d0 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47622/0 (socket says 172.21.15.7:47622) 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 -- 172.21.15.7:0/3296379378 learned_addr learned my addr 172.21.15.7:0/3296379378 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f659c700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8f80b21d0 0x7fb8f80c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f6d9d700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f8159f20 0x7fb8f815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 -- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8f80b21d0 msgr2=0x7fb8f80c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8f80b21d0 0x7fb8f80c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 -- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f8159f20 msgr2=0x7fb8f815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f8159f20 0x7fb8f815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.167+0000 7fb8f5d9b700 1 -- 172.21.15.7:0/3296379378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8009420 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8f5d9b700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c810 tx=0x7fb8ec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec005b30 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec005c90 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f815c0a0 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007300 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f80c7280 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f806cc90 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb8ec01b3b0 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 0x7fb8d8062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb8ec07c170 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.171+0000 7fb8f659c700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 0x7fb8d8062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.175+0000 7fb8f659c700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 0x7fb8d8062a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80093f0 tx=0x7fb8e8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.183+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb8ec04c270 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8f804b5c0 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8e67fc700 1 -- 172.21.15.7:0/3296379378 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8ec001d60 con 0x7fb8f80ccdb0 2023-10-06T11:22:43.347 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 msgr2=0x7fb8d8062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 0x7fb8d8062a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80093f0 tx=0x7fb8e8017000 comp rx=0 tx=0).stop 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 msgr2=0x7fb8f80b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.343+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c810 tx=0x7fb8ec007df0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 shutdown_connections 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8d80605d0 0x7fb8d8062a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8f8159f20 0x7fb8f815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f80ccdb0 0x7fb8f80b2c70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 --2- 172.21.15.7:0/3296379378 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8f80b21d0 0x7fb8f80c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 >> 172.21.15.7:0/3296379378 conn(0x7fb8f8067e30 msgr2=0x7fb8f8159710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 shutdown_connections 2023-10-06T11:22:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.347+0000 7fb8f759e700 1 -- 172.21.15.7:0/3296379378 wait complete. 2023-10-06T11:22:43.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 Processor -- start 2023-10-06T11:22:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 -- start start 2023-10-06T11:22:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6c88156d60 0x7f6c880b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 0x7f6c880b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c880cb690 con 0x7f6c88156990 2023-10-06T11:22:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c880ab320 con 0x7f6c88156d60 2023-10-06T11:22:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c8759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880cb0d0 con 0x7f6c880acaf0 2023-10-06T11:22:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6c88156d60 0x7f6c880b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c86d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 0x7f6c880b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6c88156d60 0x7f6c880b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:38288/0 (socket says 172.21.15.7:38288) 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 -- 172.21.15.7:0/972039961 learned_addr learned my addr 172.21.15.7:0/972039961 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c86d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 0x7f6c880b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:60476/0 (socket says 172.21.15.7:60476) 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 -- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 msgr2=0x7f6c880b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 --2- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 0x7f6c880b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 -- 172.21.15.7:0/972039961 >> v1:172.21.15.50:6789/0 conn(0x7f6c88156990 legacy=0x7f6c880ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.467+0000 7f6c85d9b700 1 -- 172.21.15.7:0/972039961 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880cb3b0 con 0x7f6c88156d60 2023-10-06T11:22:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c85d9b700 1 --2- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c88156d60 0x7f6c880b09c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009f20 tx=0x7f6c78009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c279ab1279d5368 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8559a700 1 -- 172.21.15.7:0/972039961 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800e070 con 0x7f6c88156d60 2023-10-06T11:22:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8559a700 1 -- 172.21.15.7:0/972039961 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7800b9d0 con 0x7f6c88156d60 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8559a700 1 -- 172.21.15.7:0/972039961 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780092e0 con 0x7f6c88156d60 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c88156d60 msgr2=0x7f6c880b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 --2- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c88156d60 0x7f6c880b09c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009f20 tx=0x7f6c78009840 comp rx=0 tx=0).stop 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- 172.21.15.7:0/972039961 shutdown_connections 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 --2- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c880acaf0 0x7f6c880b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 --2- 172.21.15.7:0/972039961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c88156d60 0x7f6c880b09c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- 172.21.15.7:0/972039961 >> 172.21.15.7:0/972039961 conn(0x7f6c880cc3c0 msgr2=0x7f6c88067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- 172.21.15.7:0/972039961 shutdown_connections 2023-10-06T11:22:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- 172.21.15.7:0/972039961 wait complete. 2023-10-06T11:22:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 Processor -- start 2023-10-06T11:22:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.471+0000 7f6c8759e700 1 -- start start 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c880acaf0 0x7f6c88150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 0x7f6c8816a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88150df0 con 0x7f6c88156d60 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8816a970 con 0x7f6c880acaf0 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8816ab80 con 0x7f6c88156990 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47628/0 (socket says 172.21.15.7:47628) 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 -- 172.21.15.7:0/3020740455 learned_addr learned my addr 172.21.15.7:0/3020740455 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c85d9b700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 0x7f6c8816a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8659c700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c880acaf0 0x7f6c88150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 -- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 msgr2=0x7f6c8816a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 0x7f6c8816a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 -- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c880acaf0 msgr2=0x7f6c88150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c880acaf0 0x7f6c88150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 -- 172.21.15.7:0/3020740455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009ae0 con 0x7f6c88156d60 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c85d9b700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 0x7f6c8816a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c86d9d700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000ea50 tx=0x7f6c800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000ee90 con 0x7f6c88156d60 2023-10-06T11:22:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8000c850 con 0x7f6c88156d60 2023-10-06T11:22:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000cc10 con 0x7f6c88156d60 2023-10-06T11:22:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8816ae60 con 0x7f6c88156d60 2023-10-06T11:22:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c88159590 con 0x7f6c88156d60 2023-10-06T11:22:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.475+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8816b360 con 0x7f6c88156d60 2023-10-06T11:22:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.479+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6c80013470 con 0x7f6c88156d60 2023-10-06T11:22:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.479+0000 7f6c767fc700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 0x7f6c68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.479+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6c8007ce40 con 0x7f6c88156d60 2023-10-06T11:22:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.479+0000 7f6c8659c700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 0x7f6c68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.479+0000 7f6c8659c700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 0x7f6c68062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c003aa0 tx=0x7f6c7c000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.487+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6c8004cf40 con 0x7f6c88156d60 2023-10-06T11:22:43.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.643+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6c88159780 con 0x7f6c88156d60 2023-10-06T11:22:43.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.643+0000 7f6c767fc700 1 -- 172.21.15.7:0/3020740455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6c80050560 con 0x7f6c88156d60 2023-10-06T11:22:43.646 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:43.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 msgr2=0x7f6c68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 0x7f6c68062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c003aa0 tx=0x7f6c7c000bc0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 msgr2=0x7f6c88143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000ea50 tx=0x7f6c800091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 shutdown_connections 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c68060550 0x7f6c68062a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c88156d60 0x7f6c88143120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6c88156990 0x7f6c8816a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 --2- 172.21.15.7:0/3020740455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c880acaf0 0x7f6c88150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 >> 172.21.15.7:0/3020740455 conn(0x7f6c880cc3c0 msgr2=0x7f6c88153710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 shutdown_connections 2023-10-06T11:22:43.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.647+0000 7f6c8759e700 1 -- 172.21.15.7:0/3020740455 wait complete. 2023-10-06T11:22:43.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 Processor -- start 2023-10-06T11:22:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 -- start start 2023-10-06T11:22:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc94155eb0 con 0x7fcc94155630 2023-10-06T11:22:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94156dc0 con 0x7fcc94157060 2023-10-06T11:22:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc98b5a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9406a870 con 0x7fcc941574e0 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc91d9b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc91d9b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:38310/0 (socket says 172.21.15.7:38310) 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:60486/0 (socket says 172.21.15.7:60486) 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc91d9b700 1 -- 172.21.15.7:0/2889697821 learned_addr learned my addr 172.21.15.7:0/2889697821 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.767+0000 7fcc9259c700 1 --1- 172.21.15.7:0/2889697821 >> v1:172.21.15.50:6789/0 conn(0x7fcc94155630 0x7fcc9406a020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:53160/0 (socket says 172.21.15.7:53160) 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc92d9d700 1 -- 172.21.15.7:0/2889697821 >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 msgr2=0x7fcc94156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc92d9d700 1 --2- 172.21.15.7:0/2889697821 >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc92d9d700 1 -- 172.21.15.7:0/2889697821 >> v1:172.21.15.50:6789/0 conn(0x7fcc94155630 legacy=0x7fcc9406a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc92d9d700 1 -- 172.21.15.7:0/2889697821 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9406ab50 con 0x7fcc941574e0 2023-10-06T11:22:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc91d9b700 1 --2- 172.21.15.7:0/2889697821 >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc9159a700 1 -- 172.21.15.7:0/2889697821 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260834409 0 0) 0x7fcc94155eb0 con 0x7fcc94155630 2023-10-06T11:22:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc92d9d700 1 --2- 172.21.15.7:0/2889697821 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009e60 tx=0x7fcc8c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5935943634dc3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc9159a700 1 -- 172.21.15.7:0/2889697821 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00e070 con 0x7fcc941574e0 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc9159a700 1 -- 172.21.15.7:0/2889697821 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00bb60 con 0x7fcc941574e0 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc9159a700 1 -- 172.21.15.7:0/2889697821 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c012430 con 0x7fcc941574e0 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- 172.21.15.7:0/2889697821 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 msgr2=0x7fcc94156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/2889697821 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009e60 tx=0x7fcc8c009780 comp rx=0 tx=0).stop 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- 172.21.15.7:0/2889697821 shutdown_connections 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/2889697821 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc941574e0 0x7fcc94156880 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/2889697821 >> v2:172.21.15.50:3300/0 conn(0x7fcc94157060 0x7fcc94156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- 172.21.15.7:0/2889697821 >> 172.21.15.7:0/2889697821 conn(0x7fcc94067e30 msgr2=0x7fcc94068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- 172.21.15.7:0/2889697821 shutdown_connections 2023-10-06T11:22:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- 172.21.15.7:0/2889697821 wait complete. 2023-10-06T11:22:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 Processor -- start 2023-10-06T11:22:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.771+0000 7fcc98b5a700 1 -- start start 2023-10-06T11:22:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc94155630 0x7fcc94165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc94157060 0x7fcc9414e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94165ab0 con 0x7fcc941574e0 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94153490 con 0x7fcc94155630 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9414e8d0 con 0x7fcc94157060 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47630/0 (socket says 172.21.15.7:47630) 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 -- 172.21.15.7:0/3117788409 learned_addr learned my addr 172.21.15.7:0/3117788409 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc91d9b700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc94157060 0x7fcc9414e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc9259c700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc94155630 0x7fcc94165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 -- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc94157060 msgr2=0x7fcc9414e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc94157060 0x7fcc9414e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 -- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc94155630 msgr2=0x7fcc94165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc94155630 0x7fcc94165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 -- 172.21.15.7:0/3117788409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c00b790 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc92d9d700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b6c0 tx=0x7fcc8c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c012e90 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c027470 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c009a20 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c027830 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9414ee40 con 0x7fcc941574e0 2023-10-06T11:22:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.775+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9404be00 con 0x7fcc941574e0 2023-10-06T11:22:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.779+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcc8c024070 con 0x7fcc941574e0 2023-10-06T11:22:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.779+0000 7fcc8a7fc700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 0x7fcc74062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.779+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcc8c07f370 con 0x7fcc941574e0 2023-10-06T11:22:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.779+0000 7fcc9259c700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 0x7fcc74062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.779+0000 7fcc9259c700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 0x7fcc74062a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ad9f0 tx=0x7fcc84007ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.787+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc8c04f470 con 0x7fcc941574e0 2023-10-06T11:22:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.947+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc94069990 con 0x7fcc941574e0 2023-10-06T11:22:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.947+0000 7fcc8a7fc700 1 -- 172.21.15.7:0/3117788409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18611 (secure 0 0 0) 0x7fcc8c052a90 con 0x7fcc941574e0 2023-10-06T11:22:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:43.954 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:01.980585+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 msgr2=0x7fcc74062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 0x7fcc74062a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ad9f0 tx=0x7fcc84007ef0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 msgr2=0x7fcc94150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b6c0 tx=0x7fcc8c019000 comp rx=0 tx=0).stop 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 shutdown_connections 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcc74060580 0x7fcc74062a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc941574e0 0x7fcc94150a10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcc94157060 0x7fcc9414e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 --2- 172.21.15.7:0/3117788409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc94155630 0x7fcc94165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 >> 172.21.15.7:0/3117788409 conn(0x7fcc94067e30 msgr2=0x7fcc940b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 shutdown_connections 2023-10-06T11:22:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:43.951+0000 7fcc98b5a700 1 -- 172.21.15.7:0/3117788409 wait complete. 2023-10-06T11:22:43.967 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:43.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.071+0000 7f0062cb2700 1 Processor -- start 2023-10-06T11:22:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.071+0000 7f0062cb2700 1 -- start start 2023-10-06T11:22:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f005c15a120 0x7f005c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c069300 con 0x7f005c06ef50 2023-10-06T11:22:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0686f0 con 0x7f005c069790 2023-10-06T11:22:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c0cd9f0 con 0x7f005c15a120 2023-10-06T11:22:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f006124f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f005c06ef50 0x7f005c0af930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52006/0 (socket says 172.21.15.7:52006) 2023-10-06T11:22:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f006124f700 1 -- 172.21.15.7:0/970836640 learned_addr learned my addr 172.21.15.7:0/970836640 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0060a4e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f005c15a120 0x7f005c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806858355 0 0) 0x7f005c069300 con 0x7f005c06ef50 2023-10-06T11:22:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004c003570 con 0x7f005c06ef50 2023-10-06T11:22:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0060a4e700 1 -- 172.21.15.7:0/970836640 >> v1:172.21.15.50:6789/0 conn(0x7f005c069790 legacy=0x7f005c06e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139158461 0 0) 0x7f005c0686f0 con 0x7f005c069790 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0060a4e700 1 -- 172.21.15.7:0/970836640 >> v1:172.21.15.7:6789/0 conn(0x7f005c06ef50 legacy=0x7f005c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0060a4e700 1 -- 172.21.15.7:0/970836640 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0689d0 con 0x7f005c15a120 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0060a4e700 1 --2- 172.21.15.7:0/970836640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c0cd4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f00500098e0 tx=0x7f00500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f385c413dbc827 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005000b640 con 0x7f005c15a120 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005000b7a0 con 0x7f005c15a120 2023-10-06T11:22:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f005b7fe700 1 -- 172.21.15.7:0/970836640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005000bb20 con 0x7f005c15a120 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- 172.21.15.7:0/970836640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 msgr2=0x7f005c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 --2- 172.21.15.7:0/970836640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c0cd4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f00500098e0 tx=0x7f00500091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- 172.21.15.7:0/970836640 shutdown_connections 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 --2- 172.21.15.7:0/970836640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c0cd4b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- 172.21.15.7:0/970836640 >> 172.21.15.7:0/970836640 conn(0x7f005c066840 msgr2=0x7f005c066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.075+0000 7f0062cb2700 1 -- 172.21.15.7:0/970836640 shutdown_connections 2023-10-06T11:22:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- 172.21.15.7:0/970836640 wait complete. 2023-10-06T11:22:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 Processor -- start 2023-10-06T11:22:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- start start 2023-10-06T11:22:44.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f005c069790 0x7f005c14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f005c06ef50 0x7f005c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c0c0360 con 0x7f005c15a120 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c0c05d0 con 0x7f005c069790 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c151380 con 0x7f005c06ef50 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47654/0 (socket says 172.21.15.7:47654) 2023-10-06T11:22:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0060a4e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f005c069790 0x7f005c14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 -- 172.21.15.7:0/297524034 learned_addr learned my addr 172.21.15.7:0/297524034 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f005bfff700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f005c06ef50 0x7f005c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 -- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f005c06ef50 msgr2=0x7f005c1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f005c06ef50 0x7f005c1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 -- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f005c069790 msgr2=0x7f005c14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f005c069790 0x7f005c14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 -- 172.21.15.7:0/297524034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0050009420 con 0x7f005c15a120 2023-10-06T11:22:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f006124f700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0054001460 tx=0x7f0054001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400a8b0 con 0x7f005c15a120 2023-10-06T11:22:44.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400aa10 con 0x7f005c15a120 2023-10-06T11:22:44.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c151660 con 0x7f005c15a120 2023-10-06T11:22:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400ad90 con 0x7f005c15a120 2023-10-06T11:22:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.079+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c0c7930 con 0x7f005c15a120 2023-10-06T11:22:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.083+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c151b60 con 0x7f005c15a120 2023-10-06T11:22:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.083+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0054007300 con 0x7f005c15a120 2023-10-06T11:22:44.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.083+0000 7f0058ff9700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 0x7f0040062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.083+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f005407cfe0 con 0x7f005c15a120 2023-10-06T11:22:44.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.087+0000 7f0060a4e700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 0x7f0040062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.087+0000 7f0060a4e700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 0x7f0040062a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00500093f0 tx=0x7f0050017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.091+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f005404d130 con 0x7f005c15a120 2023-10-06T11:22:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.255+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f005c04be00 con 0x7f005c15a120 2023-10-06T11:22:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0058ff9700 1 -- 172.21.15.7:0/297524034 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0054050750 con 0x7f005c15a120 2023-10-06T11:22:44.262 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 msgr2=0x7f0040062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 0x7f0040062a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00500093f0 tx=0x7f0050017000 comp rx=0 tx=0).stop 2023-10-06T11:22:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 msgr2=0x7f005c06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0054001460 tx=0x7f0054001a80 comp rx=0 tx=0).stop 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 shutdown_connections 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0040060550 0x7f0040062a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c15a120 0x7f005c06b8a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f005c06ef50 0x7f005c1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 --2- 172.21.15.7:0/297524034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f005c069790 0x7f005c14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 >> 172.21.15.7:0/297524034 conn(0x7f005c066840 msgr2=0x7f005c1520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 shutdown_connections 2023-10-06T11:22:44.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.259+0000 7f0062cb2700 1 -- 172.21.15.7:0/297524034 wait complete. 2023-10-06T11:22:44.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 Processor -- start 2023-10-06T11:22:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 -- start start 2023-10-06T11:22:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 0x7f717015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71700af060 0x7f71700af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717015f320 con 0x7f717015ce30 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71700afa20 con 0x7f71700af060 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f71772ba700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717015e710 con 0x7f717015e170 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71700af060 0x7f71700af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71700af060 0x7f71700af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47666/0 (socket says 172.21.15.7:47666) 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 -- 172.21.15.7:0/2804738274 learned_addr learned my addr 172.21.15.7:0/2804738274 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7175ab7700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 0x7f717015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 -- 172.21.15.7:0/2804738274 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 msgr2=0x7f717015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 0x7f717015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 -- 172.21.15.7:0/2804738274 >> v1:172.21.15.50:6789/0 conn(0x7f717015ce30 legacy=0x7f71700ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 -- 172.21.15.7:0/2804738274 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717015e9f0 con 0x7f71700af060 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7175ab7700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 0x7f717015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.383+0000 7f7176ab9700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71700af060 0x7f71700af4e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7168009f20 tx=0x7f7168009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f188694c9f6f0af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71752b6700 1 -- 172.21.15.7:0/2804738274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800e070 con 0x7f71700af060 2023-10-06T11:22:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71752b6700 1 -- 172.21.15.7:0/2804738274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800ba80 con 0x7f71700af060 2023-10-06T11:22:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71752b6700 1 -- 172.21.15.7:0/2804738274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680092e0 con 0x7f71700af060 2023-10-06T11:22:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- 172.21.15.7:0/2804738274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71700af060 msgr2=0x7f71700af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71700af060 0x7f71700af4e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7168009f20 tx=0x7f7168009840 comp rx=0 tx=0).stop 2023-10-06T11:22:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- 172.21.15.7:0/2804738274 shutdown_connections 2023-10-06T11:22:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71700af060 0x7f71700af4e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 --2- 172.21.15.7:0/2804738274 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015e170 0x7f717015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- 172.21.15.7:0/2804738274 >> 172.21.15.7:0/2804738274 conn(0x7f71700cc3c0 msgr2=0x7f7170067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- 172.21.15.7:0/2804738274 shutdown_connections 2023-10-06T11:22:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- 172.21.15.7:0/2804738274 wait complete. 2023-10-06T11:22:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 Processor -- start 2023-10-06T11:22:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.391+0000 7f71772ba700 1 -- start start 2023-10-06T11:22:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71700af060 0x7f717014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015ce30 0x7f7170170b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717014bb70 con 0x7f717015e170 2023-10-06T11:22:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717016f3f0 con 0x7f71700af060 2023-10-06T11:22:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71772ba700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717016e7e0 con 0x7f717015ce30 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71762b8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71700af060 0x7f717014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47672/0 (socket says 172.21.15.7:47672) 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7175ab7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015ce30 0x7f7170170b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 -- 172.21.15.7:0/4003836271 learned_addr learned my addr 172.21.15.7:0/4003836271 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 -- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015ce30 msgr2=0x7f7170170b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015ce30 0x7f7170170b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 -- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71700af060 msgr2=0x7f717014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71700af060 0x7f717014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 -- 172.21.15.7:0/4003836271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009ae0 con 0x7f717015e170 2023-10-06T11:22:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f7176ab9700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f716800b6c0 tx=0x7f7168018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.395+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680032f0 con 0x7f717015e170 2023-10-06T11:22:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7168008ce0 con 0x7f717015e170 2023-10-06T11:22:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717016eac0 con 0x7f717015e170 2023-10-06T11:22:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717016f080 con 0x7f717015e170 2023-10-06T11:22:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680266b0 con 0x7f717015e170 2023-10-06T11:22:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71580052b0 con 0x7f717015e170 2023-10-06T11:22:44.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.399+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7168003040 con 0x7f717015e170 2023-10-06T11:22:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.403+0000 7f71667fc700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 0x7f7154062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.403+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f716807e820 con 0x7f717015e170 2023-10-06T11:22:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.403+0000 7f71762b8700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 0x7f7154062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.403+0000 7f71762b8700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 0x7f7154062a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71700aded0 tx=0x7f716c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.407+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f716804e980 con 0x7f717015e170 2023-10-06T11:22:44.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.567+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7158005ae0 con 0x7f717015e170 2023-10-06T11:22:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.567+0000 7f71667fc700 1 -- 172.21.15.7:0/4003836271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7168051fa0 con 0x7f717015e170 2023-10-06T11:22:44.573 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:44.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 msgr2=0x7f7154062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 0x7f7154062a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71700aded0 tx=0x7f716c009040 comp rx=0 tx=0).stop 2023-10-06T11:22:44.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 msgr2=0x7f7170171050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f716800b6c0 tx=0x7f7168018000 comp rx=0 tx=0).stop 2023-10-06T11:22:44.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 shutdown_connections 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7154060550 0x7f7154062a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f717015e170 0x7f7170171050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f717015ce30 0x7f7170170b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 --2- 172.21.15.7:0/4003836271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71700af060 0x7f717014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 >> 172.21.15.7:0/4003836271 conn(0x7f71700cc3c0 msgr2=0x7f71700a58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 shutdown_connections 2023-10-06T11:22:44.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.571+0000 7f71772ba700 1 -- 172.21.15.7:0/4003836271 wait complete. 2023-10-06T11:22:44.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.695+0000 7f62d0835700 1 Processor -- start 2023-10-06T11:22:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.695+0000 7f62d0835700 1 -- start start 2023-10-06T11:22:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62d0835700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 0x7f62cc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62d0835700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc15cbb0 con 0x7f62cc15d040 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62d0835700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0caf70 con 0x7f62cc159dd0 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62d0835700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc066c10 con 0x7f62cc156880 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cb59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62cc15d040 0x7f62cc0c8740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52018/0 (socket says 172.21.15.7:52018) 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cb59e700 1 -- 172.21.15.7:0/506577820 learned_addr learned my addr 172.21.15.7:0/506577820 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cad9d700 1 --2- 172.21.15.7:0/506577820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 0x7f62cc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009190320 0 0) 0x7f62cc15cbb0 con 0x7f62cc15d040 2023-10-06T11:22:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b8003570 con 0x7f62cc15d040 2023-10-06T11:22:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370809923 0 0) 0x7f62cc0caf70 con 0x7f62cc159dd0 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62cc15cbb0 con 0x7f62cc159dd0 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cad9d700 1 -- 172.21.15.7:0/506577820 >> v1:172.21.15.50:6789/0 conn(0x7f62cc159dd0 legacy=0x7f62cc06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932423458 0 0) 0x7f62b8003570 con 0x7f62cc15d040 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cad9d700 1 -- 172.21.15.7:0/506577820 >> v1:172.21.15.7:6789/0 conn(0x7f62cc15d040 legacy=0x7f62cc0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cad9d700 1 -- 172.21.15.7:0/506577820 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0ca6d0 con 0x7f62cc156880 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62cad9d700 1 --2- 172.21.15.7:0/506577820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 0x7f62cc0c9d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009910 tx=0x7f62bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0f0e5f8eb76464 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c40017f0 con 0x7f62cc156880 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc011420 con 0x7f62cc156880 2023-10-06T11:22:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.699+0000 7f62ca59c700 1 -- 172.21.15.7:0/506577820 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0117c0 con 0x7f62cc156880 2023-10-06T11:22:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- 172.21.15.7:0/506577820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 msgr2=0x7f62cc0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 --2- 172.21.15.7:0/506577820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 0x7f62cc0c9d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009910 tx=0x7f62bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- 172.21.15.7:0/506577820 shutdown_connections 2023-10-06T11:22:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 --2- 172.21.15.7:0/506577820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc156880 0x7f62cc0c9d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- 172.21.15.7:0/506577820 >> 172.21.15.7:0/506577820 conn(0x7f62cc067e30 msgr2=0x7f62cc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- 172.21.15.7:0/506577820 shutdown_connections 2023-10-06T11:22:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- 172.21.15.7:0/506577820 wait complete. 2023-10-06T11:22:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 Processor -- start 2023-10-06T11:22:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- start start 2023-10-06T11:22:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 0x7f62cc142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc159dd0 0x7f62cc0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0c65c0 con 0x7f62cc15d040 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0ae7e0 con 0x7f62cc156880 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62d0835700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc158bd0 con 0x7f62cc159dd0 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62cb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62cb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47686/0 (socket says 172.21.15.7:47686) 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.703+0000 7f62cb59e700 1 -- 172.21.15.7:0/2453182729 learned_addr learned my addr 172.21.15.7:0/2453182729 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62c259c700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc159dd0 0x7f62cc0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cad9d700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 0x7f62cc142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 -- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc159dd0 msgr2=0x7f62cc0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc159dd0 0x7f62cc0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 -- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 msgr2=0x7f62cc142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 0x7f62cc142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 -- 172.21.15.7:0/2453182729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009420 con 0x7f62cc15d040 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cad9d700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 0x7f62cc142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62cb59e700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cc40 tx=0x7f62c4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004dc0 con 0x7f62cc15d040 2023-10-06T11:22:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400a5c0 con 0x7f62cc15d040 2023-10-06T11:22:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc1582a0 con 0x7f62cc15d040 2023-10-06T11:22:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400a980 con 0x7f62cc15d040 2023-10-06T11:22:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc158860 con 0x7f62cc15d040 2023-10-06T11:22:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.707+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc06cf60 con 0x7f62cc15d040 2023-10-06T11:22:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.711+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f62c4007150 con 0x7f62cc15d040 2023-10-06T11:22:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.711+0000 7f62c37fe700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 0x7f62ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.711+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f62c407c470 con 0x7f62cc15d040 2023-10-06T11:22:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.711+0000 7f62cad9d700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 0x7f62ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.711+0000 7f62cad9d700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 0x7f62ac062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0030c0 tx=0x7f62bc016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.715+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62c404c650 con 0x7f62cc15d040 2023-10-06T11:22:44.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62cc158d10 con 0x7f62cc15d040 2023-10-06T11:22:44.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62c37fe700 1 -- 172.21.15.7:0/2453182729 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62c4007e60 con 0x7f62cc15d040 2023-10-06T11:22:44.883 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:44.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 msgr2=0x7f62ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 0x7f62ac062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0030c0 tx=0x7f62bc016000 comp rx=0 tx=0).stop 2023-10-06T11:22:44.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 msgr2=0x7f62cc0c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:44.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.879+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cc40 tx=0x7f62c4007d20 comp rx=0 tx=0).stop 2023-10-06T11:22:44.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 shutdown_connections 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62ac060550 0x7f62ac062a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc15d040 0x7f62cc0c31e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62cc159dd0 0x7f62cc0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 --2- 172.21.15.7:0/2453182729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62cc156880 0x7f62cc142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 >> 172.21.15.7:0/2453182729 conn(0x7f62cc067e30 msgr2=0x7f62cc15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 shutdown_connections 2023-10-06T11:22:44.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:44.883+0000 7f62d0835700 1 -- 172.21.15.7:0/2453182729 wait complete. 2023-10-06T11:22:44.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:45.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 Processor -- start 2023-10-06T11:22:45.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 -- start start 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a8156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a8155d10 0x7fc3a80c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a8156620 con 0x7fc3a8155940 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a8159ff0 con 0x7fc3a8155d10 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3acf14700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a815a2d0 con 0x7fc3a806e110 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3a6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a8155d10 0x7fc3a80c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3a6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a8155d10 0x7fc3a80c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47688/0 (socket says 172.21.15.7:47688) 2023-10-06T11:22:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/1518897594 learned_addr learned my addr 172.21.15.7:0/1518897594 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.007+0000 7fc3a5d9b700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a8156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/1518897594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 msgr2=0x7fc3a8156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a6d9d700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a8156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/1518897594 >> v1:172.21.15.50:6789/0 conn(0x7fc3a8155940 legacy=0x7fc3a81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/1518897594 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815a5b0 con 0x7fc3a8155d10 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a6d9d700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c7f10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009f20 tx=0x7fc3a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965fb122ef5082e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a559a700 1 -- 172.21.15.7:0/1518897594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00092e0 con 0x7fc3a8155d10 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a559a700 1 -- 172.21.15.7:0/1518897594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a000b9d0 con 0x7fc3a8155d10 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3a559a700 1 -- 172.21.15.7:0/1518897594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000bd90 con 0x7fc3a8155d10 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 -- 172.21.15.7:0/1518897594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 msgr2=0x7fc3a80c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c7f10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009f20 tx=0x7fc3a0009840 comp rx=0 tx=0).stop 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 -- 172.21.15.7:0/1518897594 shutdown_connections 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c7f10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 --2- 172.21.15.7:0/1518897594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a8156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 -- 172.21.15.7:0/1518897594 >> 172.21.15.7:0/1518897594 conn(0x7fc3a80cc2f0 msgr2=0x7fc3a80cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 -- 172.21.15.7:0/1518897594 shutdown_connections 2023-10-06T11:22:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 -- 172.21.15.7:0/1518897594 wait complete. 2023-10-06T11:22:45.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.011+0000 7fc3acf14700 1 Processor -- start 2023-10-06T11:22:45.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- start start 2023-10-06T11:22:45.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3a8155940 0x7fc3a81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b21e0 con 0x7fc3a8155d10 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a81456f0 con 0x7fc3a8155940 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80af090 con 0x7fc3a806e110 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a659c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a659c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:60556/0 (socket says 172.21.15.7:60556) 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a659c700 1 -- 172.21.15.7:0/204403655 learned_addr learned my addr 172.21.15.7:0/204403655 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:45.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a5d9b700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3a8155940 0x7fc3a81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 msgr2=0x7fc3a80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3a8155940 msgr2=0x7fc3a81451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3a8155940 0x7fc3a81451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 -- 172.21.15.7:0/204403655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0009ae0 con 0x7fc3a8155d10 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a659c700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3a6d9d700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c5150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0017000 tx=0x7fc3a001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0020070 con 0x7fc3a8155d10 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a00032f0 con 0x7fc3a8155d10 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a80af370 con 0x7fc3a8155d10 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0011ea0 con 0x7fc3a8155d10 2023-10-06T11:22:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.015+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a80af930 con 0x7fc3a8155d10 2023-10-06T11:22:45.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.019+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a804be00 con 0x7fc3a8155d10 2023-10-06T11:22:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.019+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc3a0003040 con 0x7fc3a8155d10 2023-10-06T11:22:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.019+0000 7fc39e7fc700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 0x7fc388062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.019+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fc3a0080700 con 0x7fc3a8155d10 2023-10-06T11:22:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.023+0000 7fc3a659c700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 0x7fc388062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.023+0000 7fc3a659c700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 0x7fc388062a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc390001e90 tx=0x7fc390009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.027+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc3a00507c0 con 0x7fc3a8155d10 2023-10-06T11:22:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3a81562b0 con 0x7fc3a8155d10 2023-10-06T11:22:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc39e7fc700 1 -- 172.21.15.7:0/204403655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc3a0053de0 con 0x7fc3a8155d10 2023-10-06T11:22:45.191 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 msgr2=0x7fc388062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 0x7fc388062a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc390001e90 tx=0x7fc390009040 comp rx=0 tx=0).stop 2023-10-06T11:22:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 msgr2=0x7fc3a80c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c5150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0017000 tx=0x7fc3a001a000 comp rx=0 tx=0).stop 2023-10-06T11:22:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 shutdown_connections 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc388060580 0x7fc388062a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8155d10 0x7fc3a80c5150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3a8155940 0x7fc3a81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 --2- 172.21.15.7:0/204403655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc3a806e110 0x7fc3a80b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 >> 172.21.15.7:0/204403655 conn(0x7fc3a80cc2f0 msgr2=0x7fc3a80c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 shutdown_connections 2023-10-06T11:22:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.187+0000 7fc3acf14700 1 -- 172.21.15.7:0/204403655 wait complete. 2023-10-06T11:22:45.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 Processor -- start 2023-10-06T11:22:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 -- start start 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b40ac960 0x7f11b40acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11b4156570 0x7f11b41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b40a8dc0 con 0x7f11b40ab300 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b40ac5f0 con 0x7f11b4156570 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11ba051700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4156f00 con 0x7f11b40ac960 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11b4156570 0x7f11b41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11b40ab300 0x7f11b40ab780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52020/0 (socket says 172.21.15.7:52020) 2023-10-06T11:22:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b3fff700 1 -- 172.21.15.7:0/2640160895 learned_addr learned my addr 172.21.15.7:0/2640160895 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b37fe700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b40ac960 0x7f11b40acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2640160895 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b40ac960 msgr2=0x7f11b40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b40ac960 0x7f11b40acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2640160895 >> v1:172.21.15.7:6789/0 conn(0x7f11b40ab300 legacy=0x7f11b40ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2640160895 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b40ac030 con 0x7f11b4156570 2023-10-06T11:22:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b2ffd700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4156570 0x7f11b41569c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11a80098e0 tx=0x7f11a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cdb4a61ae9f2af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b27fc700 1 -- 172.21.15.7:0/2640160895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8009e80 con 0x7f11b4156570 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b27fc700 1 -- 172.21.15.7:0/2640160895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800b860 con 0x7f11b4156570 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.315+0000 7f11b27fc700 1 -- 172.21.15.7:0/2640160895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a800bbe0 con 0x7f11b4156570 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- 172.21.15.7:0/2640160895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4156570 msgr2=0x7f11b41569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4156570 0x7f11b41569c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11a80098e0 tx=0x7f11a80091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- 172.21.15.7:0/2640160895 shutdown_connections 2023-10-06T11:22:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4156570 0x7f11b41569c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 --2- 172.21.15.7:0/2640160895 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b40ac960 0x7f11b40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- 172.21.15.7:0/2640160895 >> 172.21.15.7:0/2640160895 conn(0x7f11b4068900 msgr2=0x7f11b40cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- 172.21.15.7:0/2640160895 shutdown_connections 2023-10-06T11:22:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- 172.21.15.7:0/2640160895 wait complete. 2023-10-06T11:22:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 Processor -- start 2023-10-06T11:22:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.319+0000 7f11ba051700 1 -- start start 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11b40ab300 0x7f11b4144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b4156570 0x7f11b415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4169920 con 0x7f11b40ac960 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b416d300 con 0x7f11b40ab300 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b416d5e0 con 0x7f11b4156570 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47728/0 (socket says 172.21.15.7:47728) 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2581257412 learned_addr learned my addr 172.21.15.7:0/2581257412 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b37fe700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11b40ab300 0x7f11b4144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b3fff700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b4156570 0x7f11b415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b4156570 msgr2=0x7f11b415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b4156570 0x7f11b415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11b40ab300 msgr2=0x7f11b4144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11b40ab300 0x7f11b4144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 -- 172.21.15.7:0/2581257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8009420 con 0x7f11b40ac960 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11b2ffd700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11b406f760 tx=0x7f11a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8000d10 con 0x7f11b40ac960 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800b9d0 con 0x7f11b40ac960 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b416d8c0 con 0x7f11b40ac960 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8025650 con 0x7f11b40ac960 2023-10-06T11:22:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.323+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b41449b0 con 0x7f11b40ac960 2023-10-06T11:22:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b406afc0 con 0x7f11b40ac960 2023-10-06T11:22:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f11a8023070 con 0x7f11b40ac960 2023-10-06T11:22:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f119b7fe700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 0x7f11a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f11a807dfc0 con 0x7f11b40ac960 2023-10-06T11:22:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f11b37fe700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 0x7f11a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.327+0000 7f11b37fe700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 0x7f11a0062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11a4006fd0 tx=0x7f11a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.339+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11a80adbe0 con 0x7f11b40ac960 2023-10-06T11:22:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f11b404be00 con 0x7f11b40ac960 2023-10-06T11:22:45.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f119b7fe700 1 -- 172.21.15.7:0/2581257412 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f11b406afc0 con 0x7f11b40ac960 2023-10-06T11:22:45.507 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 msgr2=0x7f11a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 0x7f11a0062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11a4006fd0 tx=0x7f11a4008040 comp rx=0 tx=0).stop 2023-10-06T11:22:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 msgr2=0x7f11b40c09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11b406f760 tx=0x7f11a8017000 comp rx=0 tx=0).stop 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 shutdown_connections 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f11a0060550 0x7f11a0062a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f11b4156570 0x7f11b415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b40ac960 0x7f11b40c09b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 --2- 172.21.15.7:0/2581257412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11b40ab300 0x7f11b4144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 >> 172.21.15.7:0/2581257412 conn(0x7f11b4068900 msgr2=0x7f11b4155230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 shutdown_connections 2023-10-06T11:22:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.503+0000 7f11ba051700 1 -- 172.21.15.7:0/2581257412 wait complete. 2023-10-06T11:22:45.531 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-06T11:22:45.531 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T11:22:45.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T11:22:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 Processor -- start 2023-10-06T11:22:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 -- start start 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9998159040 0x7f9998159410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9998069c10 con 0x7f99980cb880 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9998069ef0 con 0x7f999806a9d0 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999f8f3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999806a1d0 con 0x7f9998159040 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9998159040 0x7f9998159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999ce8e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99980cb880 0x7f99980cc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52028/0 (socket says 172.21.15.7:52028) 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9998159040 0x7f9998159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47732/0 (socket says 172.21.15.7:47732) 2023-10-06T11:22:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999ce8e700 1 -- 172.21.15.7:0/3659839569 learned_addr learned my addr 172.21.15.7:0/3659839569 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 -- 172.21.15.7:0/3659839569 >> v1:172.21.15.50:6789/0 conn(0x7f999806a9d0 legacy=0x7f999806d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 -- 172.21.15.7:0/3659839569 >> v1:172.21.15.7:6789/0 conn(0x7f99980cb880 legacy=0x7f99980cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f9997fff700 1 -- 172.21.15.7:0/3659839569 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312021215 0 0) 0x7f9998069c10 con 0x7f99980cb880 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 -- 172.21.15.7:0/3659839569 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999806a4b0 con 0x7f9998159040 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f999d68f700 1 --2- 172.21.15.7:0/3659839569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159040 0x7f9998159410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e60 tx=0x7f9990009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632c1d52bdc35f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f9997fff700 1 -- 172.21.15.7:0/3659839569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999000e070 con 0x7f9998159040 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f9997fff700 1 -- 172.21.15.7:0/3659839569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999000bbf0 con 0x7f9998159040 2023-10-06T11:22:45.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.631+0000 7f9997fff700 1 -- 172.21.15.7:0/3659839569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990012430 con 0x7f9998159040 2023-10-06T11:22:45.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- 172.21.15.7:0/3659839569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159040 msgr2=0x7f9998159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 --2- 172.21.15.7:0/3659839569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159040 0x7f9998159410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e60 tx=0x7f9990009780 comp rx=0 tx=0).stop 2023-10-06T11:22:45.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- 172.21.15.7:0/3659839569 shutdown_connections 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 --2- 172.21.15.7:0/3659839569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159040 0x7f9998159410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- 172.21.15.7:0/3659839569 >> 172.21.15.7:0/3659839569 conn(0x7f99980ce780 msgr2=0x7f99980ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- 172.21.15.7:0/3659839569 shutdown_connections 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- 172.21.15.7:0/3659839569 wait complete. 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 Processor -- start 2023-10-06T11:22:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- start start 2023-10-06T11:22:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99980cb880 0x7f9998156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9998159040 0x7f99981520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998156020 con 0x7f999806a9d0 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998156300 con 0x7f9998159040 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999f8f3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99981565e0 con 0x7f99980cb880 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999d68f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999d68f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47744/0 (socket says 172.21.15.7:47744) 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999d68f700 1 -- 172.21.15.7:0/2968459586 learned_addr learned my addr 172.21.15.7:0/2968459586 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999de90700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9998159040 0x7f99981520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.635+0000 7f999ce8e700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99980cb880 0x7f9998156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 -- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99980cb880 msgr2=0x7f9998156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99980cb880 0x7f9998156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 -- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9998159040 msgr2=0x7f99981520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9998159040 0x7f99981520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 -- 172.21.15.7:0/2968459586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990009a20 con 0x7f999806a9d0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999d68f700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f999806f1d0 tx=0x7f9990018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990012e80 con 0x7f999806a9d0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999001e640 con 0x7f999806a9d0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99980b0a50 con 0x7f999806a9d0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999001e9e0 con 0x7f999806a9d0 2023-10-06T11:22:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99980b36a0 con 0x7f999806a9d0 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99980b3890 con 0x7f999806a9d0 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9990024070 con 0x7f999806a9d0 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 0x7f9984062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f999007f590 con 0x7f999806a9d0 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999ce8e700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 0x7f9984062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.639+0000 7f999ce8e700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 0x7f9984062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9998155180 tx=0x7f998c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:45.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.647+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99980b3890 con 0x7f999806a9d0 2023-10-06T11:22:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:45.863+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f99980b3890 con 0x7f999806a9d0 2023-10-06T11:22:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.419+0000 7f99957fa700 1 -- 172.21.15.7:0/2968459586 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v61) v1 ==== 158+0+0 (secure 0 0 0) 0x7f99980b3890 con 0x7f999806a9d0 2023-10-06T11:22:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 msgr2=0x7f9984062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 0x7f9984062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9998155180 tx=0x7f998c009040 comp rx=0 tx=0).stop 2023-10-06T11:22:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 msgr2=0x7f99980b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f999806f1d0 tx=0x7f9990018000 comp rx=0 tx=0).stop 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 shutdown_connections 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9984060550 0x7f9984062a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9998159040 0x7f99981520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99980cb880 0x7f9998156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 --2- 172.21.15.7:0/2968459586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999806a9d0 0x7f99980b3160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 >> 172.21.15.7:0/2968459586 conn(0x7f99980ce780 msgr2=0x7f99980ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 shutdown_connections 2023-10-06T11:22:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.423+0000 7f999f8f3700 1 -- 172.21.15.7:0/2968459586 wait complete. 2023-10-06T11:22:46.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:22:46.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 Processor -- start 2023-10-06T11:22:46.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 -- start start 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc5b40b21d0 0x7fc5b40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5b4159f20 0x7fc5b40cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b415d320 con 0x7fc5b40ccbe0 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b40ccff0 con 0x7fc5b4159f20 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b359e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b415a390 con 0x7fc5b40b21d0 2023-10-06T11:22:46.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5abfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5b4159f20 0x7fc5b40cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5b40ccbe0 0x7fc5b406d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52036/0 (socket says 172.21.15.7:52036) 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.563+0000 7fc5b2d9d700 1 -- 172.21.15.7:0/1542975863 learned_addr learned my addr 172.21.15.7:0/1542975863 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b259c700 1 --2- 172.21.15.7:0/1542975863 >> v2:172.21.15.50:3300/0 conn(0x7fc5b40b21d0 0x7fc5b40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5abfff700 1 -- 172.21.15.7:0/1542975863 >> v2:172.21.15.50:3300/0 conn(0x7fc5b40b21d0 msgr2=0x7fc5b40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5abfff700 1 --2- 172.21.15.7:0/1542975863 >> v2:172.21.15.50:3300/0 conn(0x7fc5b40b21d0 0x7fc5b40ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5abfff700 1 -- 172.21.15.7:0/1542975863 >> v1:172.21.15.7:6789/0 conn(0x7fc5b40ccbe0 legacy=0x7fc5b406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5abfff700 1 -- 172.21.15.7:0/1542975863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40ce940 con 0x7fc5b4159f20 2023-10-06T11:22:46.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5abfff700 1 --2- 172.21.15.7:0/1542975863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40cc6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40cfc00 tx=0x7fc59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34824d33db5a4ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:46.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b1d9b700 1 -- 172.21.15.7:0/1542975863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c011630 con 0x7fc5b4159f20 2023-10-06T11:22:46.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b1d9b700 1 -- 172.21.15.7:0/1542975863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c011790 con 0x7fc5b4159f20 2023-10-06T11:22:46.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b1d9b700 1 -- 172.21.15.7:0/1542975863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c011b30 con 0x7fc5b4159f20 2023-10-06T11:22:46.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- 172.21.15.7:0/1542975863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 msgr2=0x7fc5b40cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 --2- 172.21.15.7:0/1542975863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40cc6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40cfc00 tx=0x7fc59c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- 172.21.15.7:0/1542975863 shutdown_connections 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 --2- 172.21.15.7:0/1542975863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40cc6a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 --2- 172.21.15.7:0/1542975863 >> v2:172.21.15.50:3300/0 conn(0x7fc5b40b21d0 0x7fc5b40ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- 172.21.15.7:0/1542975863 >> 172.21.15.7:0/1542975863 conn(0x7fc5b4067e30 msgr2=0x7fc5b4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- 172.21.15.7:0/1542975863 shutdown_connections 2023-10-06T11:22:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- 172.21.15.7:0/1542975863 wait complete. 2023-10-06T11:22:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 Processor -- start 2023-10-06T11:22:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.567+0000 7fc5b359e700 1 -- start start 2023-10-06T11:22:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5b40b21d0 0x7fc5b4148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b40ccbe0 0x7fc5b41486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b41494c0 con 0x7fc5b4159f20 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40c6df0 con 0x7fc5b40ccbe0 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40c7090 con 0x7fc5b40b21d0 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47764/0 (socket says 172.21.15.7:47764) 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 -- 172.21.15.7:0/2425072880 learned_addr learned my addr 172.21.15.7:0/2425072880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5abfff700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b40ccbe0 0x7fc5b41486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b259c700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5b40b21d0 0x7fc5b4148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 -- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5b40b21d0 msgr2=0x7fc5b4148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5b40b21d0 0x7fc5b4148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 -- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b40ccbe0 msgr2=0x7fc5b41486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b40ccbe0 0x7fc5b41486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 -- 172.21.15.7:0/2425072880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c009420 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b2d9d700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00cc40 tx=0x7fc5ac001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00a8b0 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00aa10 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b40c7370 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00add0 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.571+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b40b4180 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b406a6c0 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc5ac007300 con 0x7fc5b4159f20 2023-10-06T11:22:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5aa7fc700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 0x7fc594062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:46.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fc5ac07cf90 con 0x7fc5b4159f20 2023-10-06T11:22:46.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5b259c700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 0x7fc594062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:46.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.575+0000 7fc5b259c700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 0x7fc594062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d10 tx=0x7fc5a40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:46.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.583+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc5ac04d0d0 con 0x7fc5b4159f20 2023-10-06T11:22:46.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.747+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc5b406cc90 con 0x7fc5b4159f20 2023-10-06T11:22:46.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.755+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/2425072880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5ac0506f0 con 0x7fc5b4159f20 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 msgr2=0x7fc594062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 0x7fc594062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d10 tx=0x7fc5a40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 msgr2=0x7fc5b40b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00cc40 tx=0x7fc5ac001a50 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 shutdown_connections 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc594060550 0x7fc594062a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b4159f20 0x7fc5b40b48b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b40ccbe0 0x7fc5b41486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 --2- 172.21.15.7:0/2425072880 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc5b40b21d0 0x7fc5b4148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 >> 172.21.15.7:0/2425072880 conn(0x7fc5b4067e30 msgr2=0x7fc5b41597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 shutdown_connections 2023-10-06T11:22:46.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:46.759+0000 7fc5b359e700 1 -- 172.21.15.7:0/2425072880 wait complete. 2023-10-06T11:22:46.779 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:22:46.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:22:46.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:22:46.816 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:22:46.816 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-06T11:22:46.817 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:22:46.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T11:22:46.834 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:22:46.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:22:46.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:46.836 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:22:46.843 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-06T11:22:46.843 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:22:46.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T11:22:46.919 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:22:46.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:22:46.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:46.919 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T11:22:46.947 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-06T11:22:46.947 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:22:46.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T11:22:46.964 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:22:46.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:22:46.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:46.965 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T11:22:47.012 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-06T11:22:47.012 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:22:47.012 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T11:22:47.029 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:22:47.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:22:47.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:22:47.029 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T11:22:47.081 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:22:47.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 Processor -- start 2023-10-06T11:22:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 -- start start 2023-10-06T11:22:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f62f40c9890 0x7f62f4160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40c8cf0 con 0x7f62f40cbf30 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40c8400 con 0x7f62f40cc300 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.195+0000 7f62f989e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40c86a0 con 0x7f62f40c9890 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62f40cbf30 0x7f62f40ca230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52048/0 (socket says 172.21.15.7:52048) 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f2ffd700 1 -- 172.21.15.7:0/682800624 learned_addr learned my addr 172.21.15.7:0/682800624 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f37fe700 1 --2- 172.21.15.7:0/682800624 >> v2:172.21.15.50:3300/0 conn(0x7f62f40c9890 0x7f62f4160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904750449 0 0) 0x7f62f40c8cf0 con 0x7f62f40cbf30 2023-10-06T11:22:47.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e0003570 con 0x7f62f40cbf30 2023-10-06T11:22:47.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1197370681 0 0) 0x7f62e0003570 con 0x7f62f40cbf30 2023-10-06T11:22:47.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 >> v2:172.21.15.50:3300/0 conn(0x7f62f40c9890 msgr2=0x7f62f4160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 --2- 172.21.15.7:0/682800624 >> v2:172.21.15.50:3300/0 conn(0x7f62f40c9890 0x7f62f4160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 >> v1:172.21.15.50:6789/0 conn(0x7f62f40cc300 legacy=0x7f62f40c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40c8980 con 0x7f62f40cbf30 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f62e4001470 con 0x7f62f40cbf30 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 >> v1:172.21.15.7:6789/0 conn(0x7f62f40cbf30 legacy=0x7f62f40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62e0003a60 0x7f62e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 0x7f62e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e00051c0 0x7f62e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40c86a0 con 0x7f62e0004810 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e000c160 con 0x7f62e00051c0 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e000c320 con 0x7f62e0003a60 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 0x7f62e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f2ffd700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62e0003a60 0x7f62e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f37fe700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e00051c0 0x7f62e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 -- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62e0003a60 msgr2=0x7f62e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62e0003a60 0x7f62e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 -- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e00051c0 msgr2=0x7f62e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e00051c0 0x7f62e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f37fe700 1 -- 172.21.15.7:0/682800624 reap_dead start 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 -- 172.21.15.7:0/682800624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e0003570 con 0x7f62e0004810 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62ebfff700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 0x7f62e0004c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0018b0 tx=0x7f62dc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25133b320eda5bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc012420 con 0x7f62e0004810 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc012580 con 0x7f62e0004810 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f27fc700 1 -- 172.21.15.7:0/682800624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc012920 con 0x7f62e0004810 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f989e700 1 -- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 msgr2=0x7f62e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.199+0000 7f62f989e700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 0x7f62e0004c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0018b0 tx=0x7f62dc007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- 172.21.15.7:0/682800624 shutdown_connections 2023-10-06T11:22:47.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 --2- 172.21.15.7:0/682800624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0004810 0x7f62e0004c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- 172.21.15.7:0/682800624 >> 172.21.15.7:0/682800624 conn(0x7f62f4066840 msgr2=0x7f62f4066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:47.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- 172.21.15.7:0/682800624 shutdown_connections 2023-10-06T11:22:47.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- 172.21.15.7:0/682800624 wait complete. 2023-10-06T11:22:47.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 Processor -- start 2023-10-06T11:22:47.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- start start 2023-10-06T11:22:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e0004810 0x7f62f415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62f40c2460 0x7f62f41692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40c1a90 con 0x7f62f415a8b0 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415e6d0 con 0x7f62e0004810 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.203+0000 7f62f989e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415cdf0 con 0x7f62f40c2460 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47788/0 (socket says 172.21.15.7:47788) 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62f40c2460 0x7f62f41692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62f2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e0004810 0x7f62f415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 -- 172.21.15.7:0/1962832767 learned_addr learned my addr 172.21.15.7:0/1962832767 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 -- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62f40c2460 msgr2=0x7f62f41692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62f40c2460 0x7f62f41692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 -- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e0004810 msgr2=0x7f62f415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e0004810 0x7f62f415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 -- 172.21.15.7:0/1962832767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f415d0d0 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62ebfff700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c3f0 tx=0x7f62dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc012420 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc001200 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f415d3b0 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00ccf0 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f40c1720 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.207+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f40c1c00 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.211+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f62dc006a30 con 0x7f62f415a8b0 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.211+0000 7f62eaffd700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 0x7f62d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.211+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f62dc07e450 con 0x7f62f415a8b0 2023-10-06T11:22:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.211+0000 7f62f2ffd700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 0x7f62d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.211+0000 7f62f2ffd700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 0x7f62d4062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62e4007e90 tx=0x7f62e4001e50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.219+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62dc04e510 con 0x7f62f415a8b0 2023-10-06T11:22:47.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.423+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62f404be00 con 0x7f62f415a8b0 2023-10-06T11:22:47.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.423+0000 7f62eaffd700 1 -- 172.21.15.7:0/1962832767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f62dc051b30 con 0x7f62f415a8b0 2023-10-06T11:22:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:47.428 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 msgr2=0x7f62d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 0x7f62d4062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62e4007e90 tx=0x7f62e4001e50 comp rx=0 tx=0).stop 2023-10-06T11:22:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 msgr2=0x7f62f40c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c3f0 tx=0x7f62dc015000 comp rx=0 tx=0).stop 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 shutdown_connections 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f62d4060550 0x7f62d4062a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f62f40c2460 0x7f62f41692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f415a8b0 0x7f62f40c1f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 --2- 172.21.15.7:0/1962832767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62e0004810 0x7f62f415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 >> 172.21.15.7:0/1962832767 conn(0x7f62f4066840 msgr2=0x7f62f4143480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 shutdown_connections 2023-10-06T11:22:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.427+0000 7f62f989e700 1 -- 172.21.15.7:0/1962832767 wait complete. 2023-10-06T11:22:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:47.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:47.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 Processor -- start 2023-10-06T11:22:47.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 -- start start 2023-10-06T11:22:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c80cc720 0x7fb4c80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c806a020 con 0x7fb4c80ce300 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c80cd090 con 0x7fb4c80cc720 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8069730 con 0x7fb4c815d4b0 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.539+0000 7fb4c5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c80cc720 0x7fb4c80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c80cc720 0x7fb4c80ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47804/0 (socket says 172.21.15.7:47804) 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 -- 172.21.15.7:0/732652575 learned_addr learned my addr 172.21.15.7:0/732652575 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c659c700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 -- 172.21.15.7:0/732652575 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 msgr2=0x7fb4c806a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c806a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 -- 172.21.15.7:0/732652575 >> v1:172.21.15.7:6789/0 conn(0x7fb4c80ce300 legacy=0x7fb4c80ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 -- 172.21.15.7:0/732652575 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80699d0 con 0x7fb4c80cc720 2023-10-06T11:22:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c5d9b700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80ccb50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80099b0 tx=0x7fb4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d54f5b3f452f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c559a700 1 -- 172.21.15.7:0/732652575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8009ea0 con 0x7fb4c80cc720 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c559a700 1 -- 172.21.15.7:0/732652575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800b860 con 0x7fb4c80cc720 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c559a700 1 -- 172.21.15.7:0/732652575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800bc00 con 0x7fb4c80cc720 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- 172.21.15.7:0/732652575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 msgr2=0x7fb4c80ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80ccb50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80099b0 tx=0x7fb4b8009250 comp rx=0 tx=0).stop 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- 172.21.15.7:0/732652575 shutdown_connections 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80ccb50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 --2- 172.21.15.7:0/732652575 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c806a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- 172.21.15.7:0/732652575 >> 172.21.15.7:0/732652575 conn(0x7fb4c8067e30 msgr2=0x7fb4c8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- 172.21.15.7:0/732652575 shutdown_connections 2023-10-06T11:22:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- 172.21.15.7:0/732652575 wait complete. 2023-10-06T11:22:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 Processor -- start 2023-10-06T11:22:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.543+0000 7fb4c759e700 1 -- start start 2023-10-06T11:22:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4c80ce300 0x7fb4c80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c80b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80b2150 con 0x7fb4c80cc720 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80b2430 con 0x7fb4c80ce300 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80b2710 con 0x7fb4c815d4b0 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47808/0 (socket says 172.21.15.7:47808) 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c659c700 1 -- 172.21.15.7:0/931669727 learned_addr learned my addr 172.21.15.7:0/931669727 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c80b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c5d9b700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4c80ce300 0x7fb4c80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 -- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4c80ce300 msgr2=0x7fb4c80b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4c80ce300 0x7fb4c80b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 -- 172.21.15.7:0/931669727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 msgr2=0x7fb4c80aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 -- 172.21.15.7:0/931669727 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80094f0 con 0x7fb4c815d4b0 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c659c700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c6d9d700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c80b1ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c830 tx=0x7fb4c0001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0012630 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0012790 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0012b50 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c8146460 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.547+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c80b5310 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c80c9fa0 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb4c0011380 con 0x7fb4c815d4b0 2023-10-06T11:22:47.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4b67fc700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 0x7fb4a8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb4c007cd50 con 0x7fb4c815d4b0 2023-10-06T11:22:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4c659c700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 0x7fb4a8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.551+0000 7fb4c659c700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 0x7fb4a8062a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009fd0 tx=0x7fb4bc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.559+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb4c80c9fa0 con 0x7fb4c815d4b0 2023-10-06T11:22:47.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.727+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4c80c9fa0 con 0x7fb4c815d4b0 2023-10-06T11:22:47.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.727+0000 7fb4b67fc700 1 -- 172.21.15.7:0/931669727 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fb4c80c9fa0 con 0x7fb4c815d4b0 2023-10-06T11:22:47.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:47.734 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:45.396377+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 msgr2=0x7fb4a8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 0x7fb4a8062a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009fd0 tx=0x7fb4bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 msgr2=0x7fb4c80b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c80b1ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c830 tx=0x7fb4c0001690 comp rx=0 tx=0).stop 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 shutdown_connections 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb4a8060580 0x7fb4a8062a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb4c815d4b0 0x7fb4c80b1ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4c80ce300 0x7fb4c80b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 --2- 172.21.15.7:0/931669727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c80cc720 0x7fb4c80aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 >> 172.21.15.7:0/931669727 conn(0x7fb4c8067e30 msgr2=0x7fb4c80cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 shutdown_connections 2023-10-06T11:22:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.735+0000 7fb4c759e700 1 -- 172.21.15.7:0/931669727 wait complete. 2023-10-06T11:22:47.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:21:55.017279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:21:55.913896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T11:21:59.628649+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:22:47.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.863+0000 7f1aa359e700 1 Processor -- start 2023-10-06T11:22:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.863+0000 7f1aa359e700 1 -- start start 2023-10-06T11:22:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1aa40ce260 0x7f1aa4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa415d320 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40ce6a0 con 0x7f1aa40b21d0 2023-10-06T11:22:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa415a460 con 0x7f1aa40ce260 2023-10-06T11:22:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa1d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1aa40ccdb0 0x7f1aa40ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52058/0 (socket says 172.21.15.7:52058) 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa1d9b700 1 -- 172.21.15.7:0/4090864690 learned_addr learned my addr 172.21.15.7:0/4090864690 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa2d9d700 1 --2- 172.21.15.7:0/4090864690 >> v2:172.21.15.50:3300/0 conn(0x7f1aa40ce260 0x7f1aa4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423262724 0 0) 0x7f1aa415d320 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a8c003570 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815851190 0 0) 0x7f1aa40ce6a0 con 0x7f1aa40b21d0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aa415d320 con 0x7f1aa40b21d0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191874591 0 0) 0x7f1a8c003570 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 >> v2:172.21.15.50:3300/0 conn(0x7f1aa40ce260 msgr2=0x7f1aa4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 --2- 172.21.15.7:0/4090864690 >> v2:172.21.15.50:3300/0 conn(0x7f1aa40ce260 0x7f1aa4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 >> v1:172.21.15.50:6789/0 conn(0x7f1aa40b21d0 legacy=0x7f1aa40cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40ce940 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f1a98001470 con 0x7f1aa40ccdb0 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 >> v1:172.21.15.7:6789/0 conn(0x7f1aa40ccdb0 legacy=0x7f1aa40ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 0x7f1a8c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c004810 0x7f1a8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1a8c0051c0 0x7f1a8c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415a460 con 0x7f1a8c003bf0 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00c0b0 con 0x7f1a8c004810 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00c270 con 0x7f1a8c0051c0 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 0x7f1a8c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa1d9b700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c004810 0x7f1a8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 -- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1a8c0051c0 msgr2=0x7f1a8c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1a8c0051c0 0x7f1a8c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 -- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c004810 msgr2=0x7f1a8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c004810 0x7f1a8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 -- 172.21.15.7:0/4090864690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c003570 con 0x7f1a8c003bf0 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa2d9d700 1 -- 172.21.15.7:0/4090864690 reap_dead start 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa259c700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 0x7f1a8c003fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1a94001a10 tx=0x7f1a94007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d633a8b25a45c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400c5d0 con 0x7f1a8c003bf0 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400c730 con 0x7f1a8c003bf0 2023-10-06T11:22:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa159a700 1 -- 172.21.15.7:0/4090864690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400caf0 con 0x7f1a8c003bf0 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 msgr2=0x7f1a8c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 0x7f1a8c003fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1a94001a10 tx=0x7f1a94007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- 172.21.15.7:0/4090864690 shutdown_connections 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 --2- 172.21.15.7:0/4090864690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a8c003bf0 0x7f1a8c003fe0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- 172.21.15.7:0/4090864690 >> 172.21.15.7:0/4090864690 conn(0x7f1aa4067e30 msgr2=0x7f1aa4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.867+0000 7f1aa359e700 1 -- 172.21.15.7:0/4090864690 shutdown_connections 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 -- 172.21.15.7:0/4090864690 wait complete. 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 Processor -- start 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 -- start start 2023-10-06T11:22:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c003bf0 0x7f1aa4153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1aa416a250 0x7f1aa406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40b4c80 con 0x7f1aa40c6d50 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa406b250 con 0x7f1a8c003bf0 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa359e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40b4350 con 0x7f1aa416a250 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47830/0 (socket says 172.21.15.7:47830) 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 -- 172.21.15.7:0/550082541 learned_addr learned my addr 172.21.15.7:0/550082541 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c003bf0 0x7f1aa4153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa1d9b700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1aa416a250 0x7f1aa406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 -- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1aa416a250 msgr2=0x7f1aa406a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1aa416a250 0x7f1aa406a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 -- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c003bf0 msgr2=0x7f1aa4153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c003bf0 0x7f1aa4153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.871+0000 7f1aa2d9d700 1 -- 172.21.15.7:0/550082541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40b4630 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1aa2d9d700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c6cb0 tx=0x7f1a9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00c790 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c00f040 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa40b4910 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00cb40 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa406ae20 con 0x7f1aa40c6d50 2023-10-06T11:22:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa40b4df0 con 0x7f1aa40c6d50 2023-10-06T11:22:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1a9c014480 con 0x7f1aa40c6d50 2023-10-06T11:22:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.875+0000 7f1a927fc700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 0x7f1a84062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.879+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f1a9c07d000 con 0x7f1aa40c6d50 2023-10-06T11:22:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.879+0000 7f1aa259c700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 0x7f1a84062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.879+0000 7f1aa259c700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 0x7f1a84062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a940014d0 tx=0x7f1a94015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:47.887+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1a9c04d160 con 0x7f1aa40c6d50 2023-10-06T11:22:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1aa404b5c0 con 0x7f1aa40c6d50 2023-10-06T11:22:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1a927fc700 1 -- 172.21.15.7:0/550082541 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a9c050780 con 0x7f1aa40c6d50 2023-10-06T11:22:48.055 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 msgr2=0x7f1a84062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 0x7f1a84062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a940014d0 tx=0x7f1a94015000 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 msgr2=0x7f1aa40c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c6cb0 tx=0x7f1a9c00b040 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 shutdown_connections 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1a84060550 0x7f1a84062a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1aa40c6d50 0x7f1aa40c71f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1aa416a250 0x7f1aa406a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 --2- 172.21.15.7:0/550082541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a8c003bf0 0x7f1aa4153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 >> 172.21.15.7:0/550082541 conn(0x7f1aa4067e30 msgr2=0x7f1aa4068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.051+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 shutdown_connections 2023-10-06T11:22:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.055+0000 7f1aa359e700 1 -- 172.21.15.7:0/550082541 wait complete. 2023-10-06T11:22:48.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:48.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 Processor -- start 2023-10-06T11:22:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 -- start start 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb900b41e0 0x7fbb900b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb9006de10 0x7fbb90159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb900b1190 con 0x7fbb900b3e10 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900b0f80 con 0x7fbb900b41e0 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb95ca0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900b1470 con 0x7fbb9006de10 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb8ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb900b41e0 0x7fbb900b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb8ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb900b41e0 0x7fbb900b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47838/0 (socket says 172.21.15.7:47838) 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb8ffff700 1 -- 172.21.15.7:0/2640739593 learned_addr learned my addr 172.21.15.7:0/2640739593 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.183+0000 7fbb9549f700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb9006de10 0x7fbb90159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8ffff700 1 -- 172.21.15.7:0/2640739593 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb9006de10 msgr2=0x7fbb90159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8ffff700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb9006de10 0x7fbb90159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8ffff700 1 -- 172.21.15.7:0/2640739593 >> v1:172.21.15.50:6789/0 conn(0x7fbb900b3e10 legacy=0x7fbb9006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8ffff700 1 -- 172.21.15.7:0/2640739593 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900b1750 con 0x7fbb900b41e0 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8ffff700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb900b41e0 0x7fbb900b1f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb800099b0 tx=0x7fbb80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957a8ac9ea73e143 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8f7fe700 1 -- 172.21.15.7:0/2640739593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80009ea0 con 0x7fbb900b41e0 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8f7fe700 1 -- 172.21.15.7:0/2640739593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000b860 con 0x7fbb900b41e0 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb8f7fe700 1 -- 172.21.15.7:0/2640739593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000bc00 con 0x7fbb900b41e0 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- 172.21.15.7:0/2640739593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb900b41e0 msgr2=0x7fbb900b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb900b41e0 0x7fbb900b1f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb800099b0 tx=0x7fbb80009250 comp rx=0 tx=0).stop 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- 172.21.15.7:0/2640739593 shutdown_connections 2023-10-06T11:22:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb9006de10 0x7fbb90159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/2640739593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb900b41e0 0x7fbb900b1f50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- 172.21.15.7:0/2640739593 >> 172.21.15.7:0/2640739593 conn(0x7fbb900cfd00 msgr2=0x7fbb900d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- 172.21.15.7:0/2640739593 shutdown_connections 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- 172.21.15.7:0/2640739593 wait complete. 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 Processor -- start 2023-10-06T11:22:48.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.187+0000 7fbb95ca0700 1 -- start start 2023-10-06T11:22:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb900b3e10 0x7fbb900cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb900b41e0 0x7fbb900ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9016a050 con 0x7fbb9006de10 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90151920 con 0x7fbb900b41e0 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90151c00 con 0x7fbb900b3e10 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47848/0 (socket says 172.21.15.7:47848) 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 -- 172.21.15.7:0/1494729283 learned_addr learned my addr 172.21.15.7:0/1494729283 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb8ffff700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb900b3e10 0x7fbb900cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb9549f700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb900b41e0 0x7fbb900ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 -- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb900b3e10 msgr2=0x7fbb900cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb900b3e10 0x7fbb900cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 -- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb900b41e0 msgr2=0x7fbb900ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb900b41e0 0x7fbb900ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 -- 172.21.15.7:0/1494729283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800094f0 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb94c9e700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c590 tx=0x7fbb8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84011590 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84011710 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb90169ce0 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84011ad0 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.191+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9016aa20 con 0x7fbb9006de10 2023-10-06T11:22:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9006c940 con 0x7fbb9006de10 2023-10-06T11:22:48.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb8401b3b0 con 0x7fbb9006de10 2023-10-06T11:22:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb8cff9700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 0x7fbb7c0629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fbb8407c1d0 con 0x7fbb9006de10 2023-10-06T11:22:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb8ffff700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 0x7fbb7c0629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.195+0000 7fbb8ffff700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 0x7fbb7c0629d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb800099b0 tx=0x7fbb80017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.199+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb8404c290 con 0x7fbb9006de10 2023-10-06T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.355+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb900b1a30 con 0x7fbb9006de10 2023-10-06T11:22:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.355+0000 7fbb8cff9700 1 -- 172.21.15.7:0/1494729283 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb84017020 con 0x7fbb9006de10 2023-10-06T11:22:48.361 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 msgr2=0x7fbb7c0629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 0x7fbb7c0629d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb800099b0 tx=0x7fbb80017000 comp rx=0 tx=0).stop 2023-10-06T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 msgr2=0x7fbb9016a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c590 tx=0x7fbb8400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 shutdown_connections 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbb7c060520 0x7fbb7c0629d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb900b41e0 0x7fbb900ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbb900b3e10 0x7fbb900cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 --2- 172.21.15.7:0/1494729283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb9006de10 0x7fbb9016a4e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 >> 172.21.15.7:0/1494729283 conn(0x7fbb900cfd00 msgr2=0x7fbb9006ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 shutdown_connections 2023-10-06T11:22:48.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.359+0000 7fbb95ca0700 1 -- 172.21.15.7:0/1494729283 wait complete. 2023-10-06T11:22:48.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:22:48.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.483+0000 7f865bfff700 1 Processor -- start 2023-10-06T11:22:48.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.483+0000 7f865bfff700 1 -- start start 2023-10-06T11:22:48.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f865c155c30 0x7f865c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0caea0 con 0x7f865c06bfb0 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0c9c90 con 0x7f865c155860 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0ca570 con 0x7f865c155c30 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f865c06bfb0 0x7f865c0cb330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52060/0 (socket says 172.21.15.7:52060) 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865a7fc700 1 -- 172.21.15.7:0/759222429 learned_addr learned my addr 172.21.15.7:0/759222429 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865b7fe700 1 --2- 172.21.15.7:0/759222429 >> v2:172.21.15.50:3300/0 conn(0x7f865c155c30 0x7f865c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359580839 0 0) 0x7f865c0caea0 con 0x7f865c06bfb0 2023-10-06T11:22:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8644003570 con 0x7f865c06bfb0 2023-10-06T11:22:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354250174 0 0) 0x7f865c0c9c90 con 0x7f865c155860 2023-10-06T11:22:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c0caea0 con 0x7f865c155860 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865b7fe700 1 -- 172.21.15.7:0/759222429 >> v1:172.21.15.50:6789/0 conn(0x7f865c155860 legacy=0x7f865c06b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792437657 0 0) 0x7f8644003570 con 0x7f865c06bfb0 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865b7fe700 1 -- 172.21.15.7:0/759222429 >> v1:172.21.15.7:6789/0 conn(0x7f865c06bfb0 legacy=0x7f865c0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865b7fe700 1 -- 172.21.15.7:0/759222429 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0ca850 con 0x7f865c155c30 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865b7fe700 1 --2- 172.21.15.7:0/759222429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c155c30 0x7f865c0c8500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8654009e60 tx=0x7f8654009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53925030078f921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f865c155c30 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400b7a0 con 0x7f865c155c30 2023-10-06T11:22:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f8659ffb700 1 -- 172.21.15.7:0/759222429 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650001470 con 0x7f865c155c30 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- 172.21.15.7:0/759222429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c155c30 msgr2=0x7f865c0c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 --2- 172.21.15.7:0/759222429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c155c30 0x7f865c0c8500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8654009e60 tx=0x7f8654009780 comp rx=0 tx=0).stop 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- 172.21.15.7:0/759222429 shutdown_connections 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 --2- 172.21.15.7:0/759222429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c155c30 0x7f865c0c8500 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- 172.21.15.7:0/759222429 >> 172.21.15.7:0/759222429 conn(0x7f865c06f2a0 msgr2=0x7f865c068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- 172.21.15.7:0/759222429 shutdown_connections 2023-10-06T11:22:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.487+0000 7f865bfff700 1 -- 172.21.15.7:0/759222429 wait complete. 2023-10-06T11:22:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.491+0000 7f865bfff700 1 Processor -- start 2023-10-06T11:22:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.491+0000 7f865bfff700 1 -- start start 2023-10-06T11:22:48.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c06bfb0 0x7f865c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f865c155c30 0x7f865c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0afa40 con 0x7f865c155860 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0b22e0 con 0x7f865c06bfb0 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0b16d0 con 0x7f865c155c30 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47862/0 (socket says 172.21.15.7:47862) 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 -- 172.21.15.7:0/2848927873 learned_addr learned my addr 172.21.15.7:0/2848927873 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865b7fe700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f865c155c30 0x7f865c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865affd700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c06bfb0 0x7f865c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 -- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f865c155c30 msgr2=0x7f865c0ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f865c155c30 0x7f865c0ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 -- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c06bfb0 msgr2=0x7f865c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c06bfb0 0x7f865c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 -- 172.21.15.7:0/2848927873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009a20 con 0x7f865c155860 2023-10-06T11:22:48.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865a7fc700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f865000ed70 tx=0x7f8650001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000cac0 con 0x7f865c155860 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000cc40 con 0x7f865c155860 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650005920 con 0x7f865c155860 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c0b19b0 con 0x7f865c155860 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0b1f70 con 0x7f865c155860 2023-10-06T11:22:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.495+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c06da60 con 0x7f865c155860 2023-10-06T11:22:48.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.499+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8650005c90 con 0x7f865c155860 2023-10-06T11:22:48.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.499+0000 7f8642ffd700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 0x7f8648062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.499+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f865007f590 con 0x7f865c155860 2023-10-06T11:22:48.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.499+0000 7f865affd700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 0x7f8648062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.499+0000 7f865affd700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 0x7f8648062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f864c007fd0 tx=0x7f864c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.503+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f865004f850 con 0x7f865c155860 2023-10-06T11:22:48.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.663+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f865c06d1e0 con 0x7f865c155860 2023-10-06T11:22:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.663+0000 7f8642ffd700 1 -- 172.21.15.7:0/2848927873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8650052e70 con 0x7f865c155860 2023-10-06T11:22:48.667 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 msgr2=0x7f8648062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 0x7f8648062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f864c007fd0 tx=0x7f864c007d50 comp rx=0 tx=0).stop 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 msgr2=0x7f865c14a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f865000ed70 tx=0x7f8650001a50 comp rx=0 tx=0).stop 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 shutdown_connections 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8648060550 0x7f8648062a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f865c155c30 0x7f865c0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c155860 0x7f865c14a580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 --2- 172.21.15.7:0/2848927873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f865c06bfb0 0x7f865c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 >> 172.21.15.7:0/2848927873 conn(0x7f865c06f2a0 msgr2=0x7f865c0c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 shutdown_connections 2023-10-06T11:22:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.667+0000 7f865bfff700 1 -- 172.21.15.7:0/2848927873 wait complete. 2023-10-06T11:22:48.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:22:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 Processor -- start 2023-10-06T11:22:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 -- start start 2023-10-06T11:22:48.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f789c15cb70 0x7f789c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f789c0ae990 0x7f789c0aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0aac00 0x7f789c0ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c0ac0b0 con 0x7f789c0ae990 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c0ab1c0 con 0x7f789c15cb70 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f78a0f69700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0ab4a0 con 0x7f789c0aac00 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f789c0ae990 0x7f789c0aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f789c0ae990 0x7f789c0aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47866/0 (socket says 172.21.15.7:47866) 2023-10-06T11:22:48.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 -- 172.21.15.7:0/2361051890 learned_addr learned my addr 172.21.15.7:0/2361051890 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789b7fe700 1 --2- 172.21.15.7:0/2361051890 >> v2:172.21.15.50:3300/0 conn(0x7f789c15cb70 0x7f789c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789bfff700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0aac00 0x7f789c0ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 -- 172.21.15.7:0/2361051890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0aac00 msgr2=0x7f789c0ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0aac00 0x7f789c0ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 -- 172.21.15.7:0/2361051890 >> v2:172.21.15.50:3300/0 conn(0x7f789c15cb70 msgr2=0x7f789c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 --2- 172.21.15.7:0/2361051890 >> v2:172.21.15.50:3300/0 conn(0x7f789c15cb70 0x7f789c15cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 -- 172.21.15.7:0/2361051890 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0ab780 con 0x7f789c0ae990 2023-10-06T11:22:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.791+0000 7f789affd700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c0ae990 0x7f789c0aa6c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f789c06d050 tx=0x7f7884009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c78f83ea2026b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f789a7fc700 1 -- 172.21.15.7:0/2361051890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788400e070 con 0x7f789c0ae990 2023-10-06T11:22:48.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f789a7fc700 1 -- 172.21.15.7:0/2361051890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788400ba50 con 0x7f789c0ae990 2023-10-06T11:22:48.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f789a7fc700 1 -- 172.21.15.7:0/2361051890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840092e0 con 0x7f789c0ae990 2023-10-06T11:22:48.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 -- 172.21.15.7:0/2361051890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c0ae990 msgr2=0x7f789c0aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c0ae990 0x7f789c0aa6c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f789c06d050 tx=0x7f7884009840 comp rx=0 tx=0).stop 2023-10-06T11:22:48.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 -- 172.21.15.7:0/2361051890 shutdown_connections 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0aac00 0x7f789c0ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 --2- 172.21.15.7:0/2361051890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c0ae990 0x7f789c0aa6c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 --2- 172.21.15.7:0/2361051890 >> v2:172.21.15.50:3300/0 conn(0x7f789c15cb70 0x7f789c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 -- 172.21.15.7:0/2361051890 >> 172.21.15.7:0/2361051890 conn(0x7f789c0cc3c0 msgr2=0x7f789c067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 -- 172.21.15.7:0/2361051890 shutdown_connections 2023-10-06T11:22:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.795+0000 7f78a0f69700 1 -- 172.21.15.7:0/2361051890 wait complete. 2023-10-06T11:22:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 Processor -- start 2023-10-06T11:22:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 -- start start 2023-10-06T11:22:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 0x7f789c169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0ae990 0x7f789c146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c16a0c0 con 0x7f789c15cb70 2023-10-06T11:22:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c15b7e0 con 0x7f789c0aac00 2023-10-06T11:22:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c15abd0 con 0x7f789c0ae990 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 0x7f789c169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 0x7f789c169b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:38510/0 (socket says 172.21.15.7:38510) 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789b7fe700 1 -- 172.21.15.7:0/3235375100 learned_addr learned my addr 172.21.15.7:0/3235375100 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47868/0 (socket says 172.21.15.7:47868) 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789affd700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0ae990 0x7f789c146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 -- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0ae990 msgr2=0x7f789c146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0ae990 0x7f789c146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 -- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 msgr2=0x7f789c169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 0x7f789c169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 -- 172.21.15.7:0/3235375100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884009ae0 con 0x7f789c15cb70 2023-10-06T11:22:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f789bfff700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d890 tx=0x7f788c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c018070 con 0x7f789c15cb70 2023-10-06T11:22:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c0125f0 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c15aeb0 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.799+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c012990 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c15b470 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c06c380 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f788c014430 con 0x7f789c15cb70 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f78937fe700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 0x7f787c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f788c07cbe0 con 0x7f789c15cb70 2023-10-06T11:22:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.803+0000 7f789b7fe700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 0x7f787c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.807+0000 7f789b7fe700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 0x7f787c062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.811+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f788c04cd40 con 0x7f789c15cb70 2023-10-06T11:22:48.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.975+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f789c15b920 con 0x7f789c15cb70 2023-10-06T11:22:48.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.975+0000 7f78937fe700 1 -- 172.21.15.7:0/3235375100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f788c050360 con 0x7f789c15cb70 2023-10-06T11:22:48.980 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 msgr2=0x7f787c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 0x7f787c062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894008040 comp rx=0 tx=0).stop 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 msgr2=0x7f789c0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d890 tx=0x7f788c008040 comp rx=0 tx=0).stop 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 shutdown_connections 2023-10-06T11:22:48.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f787c060580 0x7f787c062a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f789c15cb70 0x7f789c0c3210 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f789c0ae990 0x7f789c146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 --2- 172.21.15.7:0/3235375100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f789c0aac00 0x7f789c169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 >> 172.21.15.7:0/3235375100 conn(0x7f789c0cc3c0 msgr2=0x7f789c069460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 shutdown_connections 2023-10-06T11:22:48.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:48.979+0000 7f78a0f69700 1 -- 172.21.15.7:0/3235375100 wait complete. 2023-10-06T11:22:49.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T11:22:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.115+0000 7f96fa0c9700 1 Processor -- start 2023-10-06T11:22:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.115+0000 7f96fa0c9700 1 -- start start 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.115+0000 7f96fa0c9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 0x7f96f40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.115+0000 7f96fa0c9700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 0x7f96f40ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.115+0000 7f96fa0c9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f415d320 con 0x7f96f4159f20 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f415a390 con 0x7f96f40ccdb0 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40ce940 con 0x7f96f40b21d0 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f88c6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96f4159f20 0x7f96f40cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52066/0 (socket says 172.21.15.7:52066) 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f88c6700 1 -- 172.21.15.7:0/2456679678 learned_addr learned my addr 172.21.15.7:0/2456679678 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 --2- 172.21.15.7:0/2456679678 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 0x7f96f40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f98c8700 1 --2- 172.21.15.7:0/2456679678 >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 0x7f96f40ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200933313 0 0) 0x7f96f415d320 con 0x7f96f4159f20 2023-10-06T11:22:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e0003570 con 0x7f96f4159f20 2023-10-06T11:22:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 -- 172.21.15.7:0/2456679678 >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 msgr2=0x7f96f40ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 --2- 172.21.15.7:0/2456679678 >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 0x7f96f40ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 -- 172.21.15.7:0/2456679678 >> v1:172.21.15.7:6789/0 conn(0x7f96f4159f20 legacy=0x7f96f40cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945084545 0 0) 0x7f96e0003570 con 0x7f96f4159f20 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 -- 172.21.15.7:0/2456679678 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f415c9f0 con 0x7f96f40b21d0 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f98c8700 1 --2- 172.21.15.7:0/2456679678 >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 0x7f96f40ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96f90c7700 1 --2- 172.21.15.7:0/2456679678 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 0x7f96f40ce260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009910 tx=0x7f96e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2729cf21ba66c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0001470 con 0x7f96f40b21d0 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4011420 con 0x7f96f40b21d0 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96ebfff700 1 -- 172.21.15.7:0/2456679678 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40117c0 con 0x7f96f40b21d0 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- 172.21.15.7:0/2456679678 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 msgr2=0x7f96f40ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/2456679678 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 0x7f96f40ce260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009910 tx=0x7f96e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- 172.21.15.7:0/2456679678 shutdown_connections 2023-10-06T11:22:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/2456679678 >> v2:172.21.15.50:3300/0 conn(0x7f96f40ccdb0 0x7f96f40ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/2456679678 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40b21d0 0x7f96f40ce260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- 172.21.15.7:0/2456679678 >> 172.21.15.7:0/2456679678 conn(0x7f96f4067e30 msgr2=0x7f96f4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- 172.21.15.7:0/2456679678 shutdown_connections 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.119+0000 7f96fa0c9700 1 -- 172.21.15.7:0/2456679678 wait complete. 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 Processor -- start 2023-10-06T11:22:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- start start 2023-10-06T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 0x7f96f415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 0x7f96f415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40c99d0 con 0x7f96f40b21d0 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40a8a50 con 0x7f96f4159f20 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40b2c70 con 0x7f96f40ccdb0 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47874/0 (socket says 172.21.15.7:47874) 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 -- 172.21.15.7:0/673283239 learned_addr learned my addr 172.21.15.7:0/673283239 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f88c6700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 0x7f96f415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f98c8700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 0x7f96f415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 -- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 msgr2=0x7f96f415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 0x7f96f415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 -- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 msgr2=0x7f96f415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 0x7f96f415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 -- 172.21.15.7:0/673283239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec009a20 con 0x7f96f40b21d0 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f98c8700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 0x7f96f415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f88c6700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 0x7f96f415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96f90c7700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96e40093f0 tx=0x7f96e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e401bc80 con 0x7f96f40b21d0 2023-10-06T11:22:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e401bde0 con 0x7f96f40b21d0 2023-10-06T11:22:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e4009420 con 0x7f96f40b21d0 2023-10-06T11:22:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e401c660 con 0x7f96f40b21d0 2023-10-06T11:22:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.123+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f40c9410 con 0x7f96f40b21d0 2023-10-06T11:22:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.127+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f40b2a40 con 0x7f96f40b21d0 2023-10-06T11:22:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.127+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f96e401ca90 con 0x7f96f40b21d0 2023-10-06T11:22:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.127+0000 7f96e97fa700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 0x7f96d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.127+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f96e4022030 con 0x7f96f40b21d0 2023-10-06T11:22:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.127+0000 7f96f88c6700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 0x7f96d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.131+0000 7f96f88c6700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 0x7f96d8062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96f00011e0 tx=0x7f96f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.135+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f96e404e8e0 con 0x7f96f40b21d0 2023-10-06T11:22:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f96f404b5c0 con 0x7f96f40b21d0 2023-10-06T11:22:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96e97fa700 1 -- 172.21.15.7:0/673283239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f96e402c0a0 con 0x7f96f40b21d0 2023-10-06T11:22:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 msgr2=0x7f96d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 0x7f96d8062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96f00011e0 tx=0x7f96f0009040 comp rx=0 tx=0).stop 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 msgr2=0x7f96f40c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96e40093f0 tx=0x7f96e4016000 comp rx=0 tx=0).stop 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 shutdown_connections 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f96d8060550 0x7f96d8062a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96f4159f20 0x7f96f415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f96f40ccdb0 0x7f96f415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 --2- 172.21.15.7:0/673283239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f40b21d0 0x7f96f40c8750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 >> 172.21.15.7:0/673283239 conn(0x7f96f4067e30 msgr2=0x7f96f41597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 shutdown_connections 2023-10-06T11:22:49.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.299+0000 7f96fa0c9700 1 -- 172.21.15.7:0/673283239 wait complete. 2023-10-06T11:22:49.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 Processor -- start 2023-10-06T11:22:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 -- start start 2023-10-06T11:22:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff49814fdf0 0x7ff4981501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980ae860 con 0x7ff4980cc190 2023-10-06T11:22:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980aeb40 con 0x7ff4980b07f0 2023-10-06T11:22:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49f02d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980aee20 con 0x7ff49814fdf0 2023-10-06T11:22:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49d82a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4980cc190 0x7ff4980b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52072/0 (socket says 172.21.15.7:52072) 2023-10-06T11:22:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49d82a700 1 -- 172.21.15.7:0/2362325347 learned_addr learned my addr 172.21.15.7:0/2362325347 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49e02b700 1 --2- 172.21.15.7:0/2362325347 >> v2:172.21.15.7:3300/0 conn(0x7ff49814fdf0 0x7ff4981501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49d029700 1 -- 172.21.15.7:0/2362325347 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311380467 0 0) 0x7ff4980aeb40 con 0x7ff4980b07f0 2023-10-06T11:22:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49d029700 1 -- 172.21.15.7:0/2362325347 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff484003570 con 0x7ff4980b07f0 2023-10-06T11:22:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49e02b700 1 -- 172.21.15.7:0/2362325347 >> v1:172.21.15.50:6789/0 conn(0x7ff4980b07f0 legacy=0x7ff4980aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49e02b700 1 -- 172.21.15.7:0/2362325347 >> v1:172.21.15.7:6789/0 conn(0x7ff4980cc190 legacy=0x7ff4980b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.419+0000 7ff49e02b700 1 -- 172.21.15.7:0/2362325347 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980af100 con 0x7ff49814fdf0 2023-10-06T11:22:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49e02b700 1 --2- 172.21.15.7:0/2362325347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814fdf0 0x7ff4981501c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098e0 tx=0x7ff4940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590623c1e013fdfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49d029700 1 -- 172.21.15.7:0/2362325347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400b640 con 0x7ff49814fdf0 2023-10-06T11:22:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49d029700 1 -- 172.21.15.7:0/2362325347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400b7a0 con 0x7ff49814fdf0 2023-10-06T11:22:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49d029700 1 -- 172.21.15.7:0/2362325347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400bb40 con 0x7ff49814fdf0 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 -- 172.21.15.7:0/2362325347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814fdf0 msgr2=0x7ff4981501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 --2- 172.21.15.7:0/2362325347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814fdf0 0x7ff4981501c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098e0 tx=0x7ff4940091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 -- 172.21.15.7:0/2362325347 shutdown_connections 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 --2- 172.21.15.7:0/2362325347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814fdf0 0x7ff4981501c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 -- 172.21.15.7:0/2362325347 >> 172.21.15.7:0/2362325347 conn(0x7ff4980688a0 msgr2=0x7ff4980cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 -- 172.21.15.7:0/2362325347 shutdown_connections 2023-10-06T11:22:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.423+0000 7ff49f02d700 1 -- 172.21.15.7:0/2362325347 wait complete. 2023-10-06T11:22:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 Processor -- start 2023-10-06T11:22:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 -- start start 2023-10-06T11:22:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980cc190 0x7ff498157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff49814fdf0 0x7ff498140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498154f30 con 0x7ff4980b07f0 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498153e90 con 0x7ff4980cc190 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49f02d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff498154170 con 0x7ff49814fdf0 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e02b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e02b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47894/0 (socket says 172.21.15.7:47894) 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e02b700 1 -- 172.21.15.7:0/3613383276 learned_addr learned my addr 172.21.15.7:0/3613383276 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e82c700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff49814fdf0 0x7ff498140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49d82a700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980cc190 0x7ff498157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e02b700 1 -- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff49814fdf0 msgr2=0x7ff498140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.427+0000 7ff49e02b700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff49814fdf0 0x7ff498140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49e02b700 1 -- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980cc190 msgr2=0x7ff498157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49e02b700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980cc190 0x7ff498157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49e02b700 1 -- 172.21.15.7:0/3613383276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009420 con 0x7ff4980b07f0 2023-10-06T11:22:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49e02b700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff4980af8a0 tx=0x7ff494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494000d10 con 0x7ff4980b07f0 2023-10-06T11:22:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400bb20 con 0x7ff4980b07f0 2023-10-06T11:22:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff498154450 con 0x7ff4980b07f0 2023-10-06T11:22:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494011650 con 0x7ff4980b07f0 2023-10-06T11:22:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.431+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff498154a10 con 0x7ff4980b07f0 2023-10-06T11:22:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff498154c30 con 0x7ff4980b07f0 2023-10-06T11:22:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff4940119c0 con 0x7ff4980b07f0 2023-10-06T11:22:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff48a7fc700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 0x7ff47c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7ff49407e360 con 0x7ff4980b07f0 2023-10-06T11:22:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff49d82a700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 0x7ff47c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.435+0000 7ff49d82a700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 0x7ff47c062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4981532d0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.443+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff49404e420 con 0x7ff4980b07f0 2023-10-06T11:22:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.643+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4980af3e0 con 0x7ff4980b07f0 2023-10-06T11:22:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.643+0000 7ff48a7fc700 1 -- 172.21.15.7:0/3613383276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7ff494051a40 con 0x7ff4980b07f0 2023-10-06T11:22:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:49.648 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 msgr2=0x7ff47c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 0x7ff47c062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4981532d0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 msgr2=0x7ff498153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff4980af8a0 tx=0x7ff494017000 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 shutdown_connections 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff47c060550 0x7ff47c062a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff49814fdf0 0x7ff498140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980cc190 0x7ff498157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 --2- 172.21.15.7:0/3613383276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4980b07f0 0x7ff498153540 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 >> 172.21.15.7:0/3613383276 conn(0x7ff4980688a0 msgr2=0x7ff4981477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 shutdown_connections 2023-10-06T11:22:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.647+0000 7ff49f02d700 1 -- 172.21.15.7:0/3613383276 wait complete. 2023-10-06T11:22:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:49.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:49.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.759+0000 7fefe0ac8700 1 Processor -- start 2023-10-06T11:22:49.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.759+0000 7fefe0ac8700 1 -- start start 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefdc159160 0x7fefdc0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fefdc068540 0x7fefdc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0c78b0 con 0x7fefdc158d90 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0c92f0 con 0x7fefdc159160 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0c7280 con 0x7fefdc068540 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefdc159160 0x7fefdc0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefdc159160 0x7fefdc0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47902/0 (socket says 172.21.15.7:47902) 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 -- 172.21.15.7:0/311677673 learned_addr learned my addr 172.21.15.7:0/311677673 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefdb7fe700 1 --1- 172.21.15.7:0/311677673 >> v1:172.21.15.7:6789/0 conn(0x7fefdc158d90 0x7fefdc066840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52082/0 (socket says 172.21.15.7:52082) 2023-10-06T11:22:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefdbfff700 1 --2- 172.21.15.7:0/311677673 >> v2:172.21.15.50:3300/0 conn(0x7fefdc068540 0x7fefdc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 -- 172.21.15.7:0/311677673 >> v2:172.21.15.50:3300/0 conn(0x7fefdc068540 msgr2=0x7fefdc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 --2- 172.21.15.7:0/311677673 >> v2:172.21.15.50:3300/0 conn(0x7fefdc068540 0x7fefdc0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 -- 172.21.15.7:0/311677673 >> v1:172.21.15.7:6789/0 conn(0x7fefdc158d90 legacy=0x7fefdc066840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 -- 172.21.15.7:0/311677673 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc068a40 con 0x7fefdc159160 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefd3fff700 1 --2- 172.21.15.7:0/311677673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc159160 0x7fefdc0c8870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0c82b0 tx=0x7fefc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28657aae9b1d6ead server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefdaffd700 1 -- 172.21.15.7:0/311677673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400b860 con 0x7fefdc159160 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefdaffd700 1 -- 172.21.15.7:0/311677673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc400b9c0 con 0x7fefdc159160 2023-10-06T11:22:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefdaffd700 1 -- 172.21.15.7:0/311677673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400bd60 con 0x7fefdc159160 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- 172.21.15.7:0/311677673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc159160 msgr2=0x7fefdc0c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/311677673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc159160 0x7fefdc0c8870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0c82b0 tx=0x7fefc4009250 comp rx=0 tx=0).stop 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- 172.21.15.7:0/311677673 shutdown_connections 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/311677673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc159160 0x7fefdc0c8870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/311677673 >> v2:172.21.15.50:3300/0 conn(0x7fefdc068540 0x7fefdc0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- 172.21.15.7:0/311677673 >> 172.21.15.7:0/311677673 conn(0x7fefdc06a990 msgr2=0x7fefdc067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- 172.21.15.7:0/311677673 shutdown_connections 2023-10-06T11:22:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.763+0000 7fefe0ac8700 1 -- 172.21.15.7:0/311677673 wait complete. 2023-10-06T11:22:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 Processor -- start 2023-10-06T11:22:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 -- start start 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefdc158d90 0x7fefdc0a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fefdc0aa140 0x7fefdc0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0a9cb0 con 0x7fefdc068540 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0a8e00 con 0x7fefdc158d90 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefe0ac8700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0a90a0 con 0x7fefdc0aa140 2023-10-06T11:22:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47904/0 (socket says 172.21.15.7:47904) 2023-10-06T11:22:49.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 -- 172.21.15.7:0/36405360 learned_addr learned my addr 172.21.15.7:0/36405360 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:49.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdbfff700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fefdc0aa140 0x7fefdc0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefd3fff700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefdc158d90 0x7fefdc0a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 -- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fefdc0aa140 msgr2=0x7fefdc0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fefdc0aa140 0x7fefdc0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 -- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefdc158d90 msgr2=0x7fefdc0a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefdc158d90 0x7fefdc0a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 -- 172.21.15.7:0/36405360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc40094f0 con 0x7fefdc068540 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefdb7fe700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c7c0 tx=0x7fefcc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.767+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc005a70 con 0x7fefdc068540 2023-10-06T11:22:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc0a9380 con 0x7fefdc068540 2023-10-06T11:22:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc005bd0 con 0x7fefdc068540 2023-10-06T11:22:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0070d0 con 0x7fefdc068540 2023-10-06T11:22:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc0a9940 con 0x7fefdc068540 2023-10-06T11:22:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc06d170 con 0x7fefdc068540 2023-10-06T11:22:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fefcc007480 con 0x7fefdc068540 2023-10-06T11:22:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd37fe700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 0x7fefbc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fefcc07c120 con 0x7fefdc068540 2023-10-06T11:22:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.771+0000 7fefd3fff700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 0x7fefbc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.775+0000 7fefd3fff700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 0x7fefbc062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0c82b0 tx=0x7fefc4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:49.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.779+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fefdc06d170 con 0x7fefdc068540 2023-10-06T11:22:49.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.943+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefdc06d170 con 0x7fefdc068540 2023-10-06T11:22:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.943+0000 7fefd37fe700 1 -- 172.21.15.7:0/36405360 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fefdc06d170 con 0x7fefdc068540 2023-10-06T11:22:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:49.949 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:45.396377+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 msgr2=0x7fefbc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 0x7fefbc062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0c82b0 tx=0x7fefc4017000 comp rx=0 tx=0).stop 2023-10-06T11:22:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 msgr2=0x7fefdc0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c7c0 tx=0x7fefcc007f70 comp rx=0 tx=0).stop 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 shutdown_connections 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fefbc060550 0x7fefbc062a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fefdc0aa140 0x7fefdc0ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefdc158d90 0x7fefdc0a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 --2- 172.21.15.7:0/36405360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefdc068540 0x7fefdc0b1a10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 >> 172.21.15.7:0/36405360 conn(0x7fefdc06a990 msgr2=0x7fefdc067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 shutdown_connections 2023-10-06T11:22:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:49.947+0000 7fefe0ac8700 1 -- 172.21.15.7:0/36405360 wait complete. 2023-10-06T11:22:49.963 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:22:49.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 Processor -- start 2023-10-06T11:22:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 -- start start 2023-10-06T11:22:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7bb414c200 0x7f7bb406e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bb40b1000 0x7f7bb4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4159850 con 0x7f7bb41563a0 2023-10-06T11:22:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb4156840 con 0x7f7bb40b1000 2023-10-06T11:22:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb93fd700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb406e890 con 0x7f7bb414c200 2023-10-06T11:22:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bb40b1000 0x7f7bb4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb2ffd700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7bb414c200 0x7f7bb406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bb40b1000 0x7f7bb4156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47920/0 (socket says 172.21.15.7:47920) 2023-10-06T11:22:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.067+0000 7f7bb37fe700 1 -- 172.21.15.7:0/427253864 learned_addr learned my addr 172.21.15.7:0/427253864 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb37fe700 1 -- 172.21.15.7:0/427253864 >> v2:172.21.15.50:3300/0 conn(0x7f7bb414c200 msgr2=0x7f7bb406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb37fe700 1 --2- 172.21.15.7:0/427253864 >> v2:172.21.15.50:3300/0 conn(0x7f7bb414c200 0x7f7bb406e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb37fe700 1 -- 172.21.15.7:0/427253864 >> v1:172.21.15.50:6789/0 conn(0x7f7bb41563a0 legacy=0x7f7bb40b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb37fe700 1 -- 172.21.15.7:0/427253864 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4158f20 con 0x7f7bb40b1000 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb37fe700 1 --2- 172.21.15.7:0/427253864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb4156a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7bb406bab0 tx=0x7f7bac00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce53cdd0068e5241 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb27fc700 1 -- 172.21.15.7:0/427253864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00f070 con 0x7f7bb40b1000 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb27fc700 1 -- 172.21.15.7:0/427253864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00a2e0 con 0x7f7bb40b1000 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb27fc700 1 -- 172.21.15.7:0/427253864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00cd50 con 0x7f7bb40b1000 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 -- 172.21.15.7:0/427253864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 msgr2=0x7f7bb4156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/427253864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb4156a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7bb406bab0 tx=0x7f7bac00a950 comp rx=0 tx=0).stop 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 -- 172.21.15.7:0/427253864 shutdown_connections 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/427253864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb4156a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/427253864 >> v2:172.21.15.50:3300/0 conn(0x7f7bb414c200 0x7f7bb406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 -- 172.21.15.7:0/427253864 >> 172.21.15.7:0/427253864 conn(0x7f7bb4066840 msgr2=0x7f7bb4066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 -- 172.21.15.7:0/427253864 shutdown_connections 2023-10-06T11:22:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.071+0000 7f7bb93fd700 1 -- 172.21.15.7:0/427253864 wait complete. 2023-10-06T11:22:50.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 Processor -- start 2023-10-06T11:22:50.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 -- start start 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb414c200 0x7f7bb4155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bb41563a0 0x7f7bb414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb414d070 con 0x7f7bb40b1000 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb414d350 con 0x7f7bb414c200 2023-10-06T11:22:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb93fd700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb414d630 con 0x7f7bb41563a0 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47930/0 (socket says 172.21.15.7:47930) 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 -- 172.21.15.7:0/23063302 learned_addr learned my addr 172.21.15.7:0/23063302 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7babfff700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb414c200 0x7f7bb4155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb37fe700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bb41563a0 0x7f7bb414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 -- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bb41563a0 msgr2=0x7f7bb414c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bb41563a0 0x7f7bb414c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 -- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb414c200 msgr2=0x7f7bb4155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb414c200 0x7f7bb4155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.075+0000 7f7bb2ffd700 1 -- 172.21.15.7:0/23063302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac00abf0 con 0x7f7bb40b1000 2023-10-06T11:22:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7bb2ffd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4010e90 tx=0x7f7ba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400ed40 con 0x7f7bb40b1000 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4011450 con 0x7f7bb40b1000 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb414d910 con 0x7f7bb40b1000 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400eea0 con 0x7f7bb40b1000 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41556c0 con 0x7f7bb40b1000 2023-10-06T11:22:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb40ca510 con 0x7f7bb40b1000 2023-10-06T11:22:50.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.079+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7ba4011a00 con 0x7f7bb40b1000 2023-10-06T11:22:50.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.083+0000 7f7baaffd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 0x7f7b94062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.083+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f7ba407f030 con 0x7f7bb40b1000 2023-10-06T11:22:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.083+0000 7f7babfff700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 0x7f7b94062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.083+0000 7f7babfff700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 0x7f7b94062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4158630 tx=0x7f7b9c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.087+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7bb40ca510 con 0x7f7bb40b1000 2023-10-06T11:22:50.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb40ca510 con 0x7f7bb40b1000 2023-10-06T11:22:50.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7baaffd700 1 -- 172.21.15.7:0/23063302 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7f7bb40ca510 con 0x7f7bb40b1000 2023-10-06T11:22:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:50.287 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:50.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 msgr2=0x7f7b94062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 0x7f7b94062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4158630 tx=0x7f7b9c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 msgr2=0x7f7bb414e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4010e90 tx=0x7f7ba40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 shutdown_connections 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b94060550 0x7f7b94062a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bb41563a0 0x7f7bb414c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb414c200 0x7f7bb4155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 --2- 172.21.15.7:0/23063302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb40b1000 0x7f7bb414e110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 >> 172.21.15.7:0/23063302 conn(0x7f7bb4066840 msgr2=0x7f7bb40aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.283+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 shutdown_connections 2023-10-06T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.287+0000 7f7bb93fd700 1 -- 172.21.15.7:0/23063302 wait complete. 2023-10-06T11:22:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:50.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:22:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 Processor -- start 2023-10-06T11:22:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 -- start start 2023-10-06T11:22:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcd1c0b02a0 0x7fcd1c0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c0c8610 0x7fcd1c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c0aba80 con 0x7fcd1c155070 2023-10-06T11:22:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c0af7f0 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd20fa2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0acf20 con 0x7fcd1c0c8610 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd1b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcd1c155070 0x7fcd1c0afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52098/0 (socket says 172.21.15.7:52098) 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd1b7fe700 1 -- 172.21.15.7:0/3886431289 learned_addr learned my addr 172.21.15.7:0/3886431289 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3886431289 >> v2:172.21.15.50:3300/0 conn(0x7fcd1c0b02a0 0x7fcd1c0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c0c8610 0x7fcd1c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.403+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/3886431289 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110050534 0 0) 0x7fcd1c0aba80 con 0x7fcd1c155070 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/3886431289 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd04004680 con 0x7fcd1c155070 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 -- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c0c8610 msgr2=0x7fcd1c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c0c8610 0x7fcd1c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 -- 172.21.15.7:0/3886431289 >> v1:172.21.15.7:6789/0 conn(0x7fcd1c155070 legacy=0x7fcd1c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 -- 172.21.15.7:0/3886431289 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0af510 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c0ae7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd100098e0 tx=0x7fcd100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44a18b1e811b41c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/3886431289 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10011630 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/3886431289 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10011790 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/3886431289 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10011b30 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 msgr2=0x7fcd1c0ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c0ae7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd100098e0 tx=0x7fcd100091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- 172.21.15.7:0/3886431289 shutdown_connections 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c0c8610 0x7fcd1c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/3886431289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c0ae7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- 172.21.15.7:0/3886431289 >> 172.21.15.7:0/3886431289 conn(0x7fcd1c0688a0 msgr2=0x7fcd1c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- 172.21.15.7:0/3886431289 shutdown_connections 2023-10-06T11:22:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- 172.21.15.7:0/3886431289 wait complete. 2023-10-06T11:22:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 Processor -- start 2023-10-06T11:22:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- start start 2023-10-06T11:22:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c155070 0x7fcd1c0a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0a8bf0 con 0x7fcd1c0c8610 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c156e90 con 0x7fcd1c0b02a0 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd20fa2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c156280 con 0x7fcd1c155070 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47944/0 (socket says 172.21.15.7:47944) 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1affd700 1 -- 172.21.15.7:0/2248173427 learned_addr learned my addr 172.21.15.7:0/2248173427 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.407+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c155070 0x7fcd1c0a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 -- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c155070 msgr2=0x7fcd1c0a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c155070 0x7fcd1c0a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 -- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 msgr2=0x7fcd1c157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 -- 172.21.15.7:0/2248173427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009420 con 0x7fcd1c0c8610 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1affd700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd100030c0 tx=0x7fcd10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10000d10 con 0x7fcd1c0c8610 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10020070 con 0x7fcd1c0c8610 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c156560 con 0x7fcd1c0c8610 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100276b0 con 0x7fcd1c0c8610 2023-10-06T11:22:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c156b20 con 0x7fcd1c0c8610 2023-10-06T11:22:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c157000 con 0x7fcd1c0c8610 2023-10-06T11:22:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcd1001c450 con 0x7fcd1c0c8610 2023-10-06T11:22:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 0x7fcd08062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fcd10022070 con 0x7fcd1c0c8610 2023-10-06T11:22:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.411+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 0x7fcd08062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.415+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 0x7fcd08062a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c007f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.419+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd1004ecf0 con 0x7fcd1c0c8610 2023-10-06T11:22:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd1c06da60 con 0x7fcd1c0c8610 2023-10-06T11:22:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd037fe700 1 -- 172.21.15.7:0/2248173427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4584 (secure 0 0 0) 0x7fcd10052310 con 0x7fcd1c0c8610 2023-10-06T11:22:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:50.623 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3161955854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3161955854},{"type":"v1","addr":"172.21.15.50:6835","nonce":3161955854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2159413998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2159413998},{"type":"v1","addr":"172.21.15.50:6837","nonce":2159413998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2021297075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2021297075},{"type":"v1","addr":"172.21.15.7:6833","nonce":2021297075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:21:56.923085+0000","modified":"2023-10-06T11:22:16.238076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/932941616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":932941616},{"type":"v1","addr":"172.21.15.7:6835","nonce":932941616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 msgr2=0x7fcd08062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 0x7fcd08062a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c007f30 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 msgr2=0x7fcd1c154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd100030c0 tx=0x7fcd10016000 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 shutdown_connections 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcd08060580 0x7fcd08062a30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcd1c155070 0x7fcd1c0a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0c8610 0x7fcd1c154310 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 --2- 172.21.15.7:0/2248173427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd1c0b02a0 0x7fcd1c157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 >> 172.21.15.7:0/2248173427 conn(0x7fcd1c0688a0 msgr2=0x7fcd1c06ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 shutdown_connections 2023-10-06T11:22:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.619+0000 7fcd20fa2700 1 -- 172.21.15.7:0/2248173427 wait complete. 2023-10-06T11:22:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-10-06T11:22:50.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.739+0000 7fa6d6921700 1 Processor -- start 2023-10-06T11:22:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.739+0000 7fa6d6921700 1 -- start start 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6d015ab80 0x7fa6d00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6d015af50 0x7fa6d00af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d015b3c0 con 0x7fa6d00aec70 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d015f540 con 0x7fa6d015ab80 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d015e950 con 0x7fa6d015af50 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6d015ab80 0x7fa6d00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6d015ab80 0x7fa6d00ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47956/0 (socket says 172.21.15.7:47956) 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 -- 172.21.15.7:0/2394989213 learned_addr learned my addr 172.21.15.7:0/2394989213 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d511e700 1 --2- 172.21.15.7:0/2394989213 >> v2:172.21.15.50:3300/0 conn(0x7fa6d015af50 0x7fa6d00af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 -- 172.21.15.7:0/2394989213 >> v2:172.21.15.50:3300/0 conn(0x7fa6d015af50 msgr2=0x7fa6d00af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 --2- 172.21.15.7:0/2394989213 >> v2:172.21.15.50:3300/0 conn(0x7fa6d015af50 0x7fa6d00af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 -- 172.21.15.7:0/2394989213 >> v1:172.21.15.50:6789/0 conn(0x7fa6d00aec70 legacy=0x7fa6d0160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 -- 172.21.15.7:0/2394989213 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d015ec10 con 0x7fa6d015ab80 2023-10-06T11:22:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d591f700 1 --2- 172.21.15.7:0/2394989213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d015ab80 0x7fa6d00ae730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cb900 tx=0x7fa6cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38ac9e2cb6ff7ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d491d700 1 -- 172.21.15.7:0/2394989213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc009e70 con 0x7fa6d015ab80 2023-10-06T11:22:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d491d700 1 -- 172.21.15.7:0/2394989213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00b860 con 0x7fa6d015ab80 2023-10-06T11:22:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d491d700 1 -- 172.21.15.7:0/2394989213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00bc00 con 0x7fa6d015ab80 2023-10-06T11:22:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- 172.21.15.7:0/2394989213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d015ab80 msgr2=0x7fa6d00ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2394989213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d015ab80 0x7fa6d00ae730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cb900 tx=0x7fa6cc009250 comp rx=0 tx=0).stop 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- 172.21.15.7:0/2394989213 shutdown_connections 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2394989213 >> v2:172.21.15.50:3300/0 conn(0x7fa6d015af50 0x7fa6d00af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2394989213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d015ab80 0x7fa6d00ae730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- 172.21.15.7:0/2394989213 >> 172.21.15.7:0/2394989213 conn(0x7fa6d00688a0 msgr2=0x7fa6d00cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- 172.21.15.7:0/2394989213 shutdown_connections 2023-10-06T11:22:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.743+0000 7fa6d6921700 1 -- 172.21.15.7:0/2394989213 wait complete. 2023-10-06T11:22:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.747+0000 7fa6d6921700 1 Processor -- start 2023-10-06T11:22:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.747+0000 7fa6d6921700 1 -- start start 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa6d015ab80 0x7fa6d00c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d015af50 0x7fa6d00c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00aa3d0 con 0x7fa6d00aec70 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00cb730 con 0x7fa6d015af50 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00a97c0 con 0x7fa6d015ab80 2023-10-06T11:22:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47958/0 (socket says 172.21.15.7:47958) 2023-10-06T11:22:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 -- 172.21.15.7:0/2822401132 learned_addr learned my addr 172.21.15.7:0/2822401132 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d511e700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa6d015ab80 0x7fa6d00c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6120700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d015af50 0x7fa6d00c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 -- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa6d015ab80 msgr2=0x7fa6d00c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa6d015ab80 0x7fa6d00c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 -- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d015af50 msgr2=0x7fa6d00c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d015af50 0x7fa6d00c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 -- 172.21.15.7:0/2822401132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc0094f0 con 0x7fa6d00aec70 2023-10-06T11:22:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d591f700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006fd0 tx=0x7fa6cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00bbe0 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc011420 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0117e0 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d00a9aa0 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d00aa060 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.751+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d00adfe0 con 0x7fa6d00aec70 2023-10-06T11:22:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.755+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa6cc011c10 con 0x7fa6d00aec70 2023-10-06T11:22:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.755+0000 7fa6c1ffb700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 0x7fa6b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:50.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.755+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fa6cc07e430 con 0x7fa6d00aec70 2023-10-06T11:22:50.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.755+0000 7fa6d511e700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 0x7fa6b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:50.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.755+0000 7fa6d511e700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 0x7fa6b4062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00adea0 tx=0x7fa6c40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:50.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.763+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa6d00adfe0 con 0x7fa6d00aec70 2023-10-06T11:22:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.927+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6d00adfe0 con 0x7fa6d00aec70 2023-10-06T11:22:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/2822401132 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fa6d00adfe0 con 0x7fa6d00aec70 2023-10-06T11:22:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:50.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:45.396377+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:21:55.017279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:21:55.913896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T11:21:59.628649+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1b","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 msgr2=0x7fa6b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 0x7fa6b4062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00adea0 tx=0x7fa6c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 msgr2=0x7fa6d00cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006fd0 tx=0x7fa6cc017000 comp rx=0 tx=0).stop 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 shutdown_connections 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa6b4060550 0x7fa6b4062a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6d015af50 0x7fa6d00c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa6d015ab80 0x7fa6d00c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 --2- 172.21.15.7:0/2822401132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d00aec70 0x7fa6d00cb1f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 >> 172.21.15.7:0/2822401132 conn(0x7fa6d00688a0 msgr2=0x7fa6d0069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 shutdown_connections 2023-10-06T11:22:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:50.931+0000 7fa6d6921700 1 -- 172.21.15.7:0/2822401132 wait complete. 2023-10-06T11:22:50.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:22:51.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.055+0000 7f79fb59e700 1 Processor -- start 2023-10-06T11:22:51.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.055+0000 7f79fb59e700 1 -- start start 2023-10-06T11:22:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc1569d0 0x7f79fc156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79fc0cc550 0x7f79fc0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79fc0b0a40 0x7f79fc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0b13e0 con 0x7f79fc0cc550 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0c8800 con 0x7f79fc0b0a40 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0cabe0 con 0x7f79fc1569d0 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79fc0cc550 0x7f79fc0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79fc0cc550 0x7f79fc0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47960/0 (socket says 172.21.15.7:47960) 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 -- 172.21.15.7:0/2580318749 learned_addr learned my addr 172.21.15.7:0/2580318749 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fa59c700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc1569d0 0x7f79fc156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fad9d700 1 --2- 172.21.15.7:0/2580318749 >> v2:172.21.15.50:3300/0 conn(0x7f79fc0b0a40 0x7f79fc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 -- 172.21.15.7:0/2580318749 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc1569d0 msgr2=0x7f79fc156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc1569d0 0x7f79fc156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 -- 172.21.15.7:0/2580318749 >> v2:172.21.15.50:3300/0 conn(0x7f79fc0b0a40 msgr2=0x7f79fc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 --2- 172.21.15.7:0/2580318749 >> v2:172.21.15.50:3300/0 conn(0x7f79fc0b0a40 0x7f79fc0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 -- 172.21.15.7:0/2580318749 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0caec0 con 0x7f79fc0cc550 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f3fff700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc0cc550 0x7f79fc0c82c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f79fc06e8c0 tx=0x7f79e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9734ad822a0655de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f9d9b700 1 -- 172.21.15.7:0/2580318749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400b640 con 0x7f79fc0cc550 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f9d9b700 1 -- 172.21.15.7:0/2580318749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400b7a0 con 0x7f79fc0cc550 2023-10-06T11:22:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79f9d9b700 1 -- 172.21.15.7:0/2580318749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400bb60 con 0x7f79fc0cc550 2023-10-06T11:22:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- 172.21.15.7:0/2580318749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc0cc550 msgr2=0x7f79fc0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc0cc550 0x7f79fc0c82c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f79fc06e8c0 tx=0x7f79e4009250 comp rx=0 tx=0).stop 2023-10-06T11:22:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- 172.21.15.7:0/2580318749 shutdown_connections 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- 172.21.15.7:0/2580318749 >> v2:172.21.15.50:3300/0 conn(0x7f79fc0b0a40 0x7f79fc0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc0cc550 0x7f79fc0c82c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 --2- 172.21.15.7:0/2580318749 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc1569d0 0x7f79fc156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- 172.21.15.7:0/2580318749 >> 172.21.15.7:0/2580318749 conn(0x7f79fc0688a0 msgr2=0x7f79fc0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- 172.21.15.7:0/2580318749 shutdown_connections 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.059+0000 7f79fb59e700 1 -- 172.21.15.7:0/2580318749 wait complete. 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 Processor -- start 2023-10-06T11:22:51.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- start start 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79fc0b0a40 0x7f79fc1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc0cc550 0x7f79fc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c8320 con 0x7f79fc1569d0 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c28f0 con 0x7f79fc0b0a40 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c22e0 con 0x7f79fc0cc550 2023-10-06T11:22:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47966/0 (socket says 172.21.15.7:47966) 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 -- 172.21.15.7:0/3038244450 learned_addr learned my addr 172.21.15.7:0/3038244450 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fa59c700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79fc0b0a40 0x7f79fc1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79f3fff700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc0cc550 0x7f79fc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 -- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc0cc550 msgr2=0x7f79fc0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc0cc550 0x7f79fc0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 -- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79fc0b0a40 msgr2=0x7f79fc1454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79fc0b0a40 0x7f79fc1454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 -- 172.21.15.7:0/3038244450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40094f0 con 0x7f79fc1569d0 2023-10-06T11:22:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fad9d700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009e90 tx=0x7f79f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400cb70 con 0x7f79fc1569d0 2023-10-06T11:22:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f400f040 con 0x7f79fc1569d0 2023-10-06T11:22:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc0c8040 con 0x7f79fc1569d0 2023-10-06T11:22:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4014480 con 0x7f79fc1569d0 2023-10-06T11:22:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.063+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc145a30 con 0x7f79fc1569d0 2023-10-06T11:22:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc0c2140 con 0x7f79fc1569d0 2023-10-06T11:22:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f79f40147f0 con 0x7f79fc1569d0 2023-10-06T11:22:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79f27fc700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 0x7f79dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f79f407dfe0 con 0x7f79fc1569d0 2023-10-06T11:22:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79fa59c700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 0x7f79dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.067+0000 7f79fa59c700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 0x7f79dc062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f79fc153ac0 tx=0x7f79ec0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.075+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f79f404e160 con 0x7f79fc1569d0 2023-10-06T11:22:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.271+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f79fc0c2ab0 con 0x7f79fc1569d0 2023-10-06T11:22:51.275 INFO:tasks.ceph.mon.a.smithi007.stderr:2023-10-06T11:22:51.271+0000 7f67b417b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:22:51.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79f27fc700 1 -- 172.21.15.7:0/3038244450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f79f4018090 con 0x7f79fc1569d0 2023-10-06T11:22:51.288 INFO:tasks.ceph.mds.a.smithi007.stderr: -3339> 2023-10-06T11:22:02.287+0000 7f86cfcc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 msgr2=0x7f79dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 0x7f79dc062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f79fc153ac0 tx=0x7f79ec0091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 msgr2=0x7f79fc15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009e90 tx=0x7f79f400b040 comp rx=0 tx=0).stop 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 shutdown_connections 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f79dc060550 0x7f79dc062a00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79fc1569d0 0x7f79fc15b990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f79fc0cc550 0x7f79fc0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 --2- 172.21.15.7:0/3038244450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79fc0b0a40 0x7f79fc1454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 >> 172.21.15.7:0/3038244450 conn(0x7f79fc0688a0 msgr2=0x7f79fc06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 shutdown_connections 2023-10-06T11:22:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.283+0000 7f79fb59e700 1 -- 172.21.15.7:0/3038244450 wait complete. 2023-10-06T11:22:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:22:51.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:22:51.315 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-06T11:22:51.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 Processor -- start 2023-10-06T11:22:51.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 -- start start 2023-10-06T11:22:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa040ca890 0x7ffa040c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffa0406a4a0 0x7ffa04158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa040cadd0 con 0x7ffa040c8970 2023-10-06T11:22:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0406a010 con 0x7ffa040ca890 2023-10-06T11:22:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa040696e0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa040ca890 0x7ffa040c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa040ca890 0x7ffa040c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47988/0 (socket says 172.21.15.7:47988) 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 -- 172.21.15.7:0/3066339725 learned_addr learned my addr 172.21.15.7:0/3066339725 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0a8c2700 1 --2- 172.21.15.7:0/3066339725 >> v2:172.21.15.50:3300/0 conn(0x7ffa0406a4a0 0x7ffa04158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 -- 172.21.15.7:0/3066339725 >> v2:172.21.15.50:3300/0 conn(0x7ffa0406a4a0 msgr2=0x7ffa04158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 --2- 172.21.15.7:0/3066339725 >> v2:172.21.15.50:3300/0 conn(0x7ffa0406a4a0 0x7ffa04158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 -- 172.21.15.7:0/3066339725 >> v1:172.21.15.50:6789/0 conn(0x7ffa040c8970 legacy=0x7ffa040ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 -- 172.21.15.7:0/3066339725 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040699c0 con 0x7ffa040ca890 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa098c0700 1 --2- 172.21.15.7:0/3066339725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa040ca890 0x7ffa040c8d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa04064240 tx=0x7ff9f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce601d9256ede807 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa090bf700 1 -- 172.21.15.7:0/3066339725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400b860 con 0x7ffa040ca890 2023-10-06T11:22:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa090bf700 1 -- 172.21.15.7:0/3066339725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400b9c0 con 0x7ffa040ca890 2023-10-06T11:22:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa090bf700 1 -- 172.21.15.7:0/3066339725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400bd60 con 0x7ffa040ca890 2023-10-06T11:22:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/3066339725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa040ca890 msgr2=0x7ffa040c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.403+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/3066339725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa040ca890 0x7ffa040c8d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa04064240 tx=0x7ff9f4009250 comp rx=0 tx=0).stop 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/3066339725 shutdown_connections 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/3066339725 >> v2:172.21.15.50:3300/0 conn(0x7ffa0406a4a0 0x7ffa04158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/3066339725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa040ca890 0x7ffa040c8d40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/3066339725 >> 172.21.15.7:0/3066339725 conn(0x7ffa04067e30 msgr2=0x7ffa04068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/3066339725 shutdown_connections 2023-10-06T11:22:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/3066339725 wait complete. 2023-10-06T11:22:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 Processor -- start 2023-10-06T11:22:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.407+0000 7ffa0b0c3700 1 -- start start 2023-10-06T11:22:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa040c8970 0x7ffa04169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffa040ca890 0x7ffa040b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041588e0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040b0160 con 0x7ffa040c8970 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0416ca40 con 0x7ffa040ca890 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48000/0 (socket says 172.21.15.7:48000) 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 -- 172.21.15.7:0/848258339 learned_addr learned my addr 172.21.15.7:0/848258339 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa098c0700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa040c8970 0x7ffa04169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a8c2700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffa040ca890 0x7ffa040b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 -- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffa040ca890 msgr2=0x7ffa040b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffa040ca890 0x7ffa040b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 -- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa040c8970 msgr2=0x7ffa04169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa040c8970 0x7ffa04169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 -- 172.21.15.7:0/848258339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f40094f0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0a0c1700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c590 tx=0x7ffa0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00011590 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa00011710 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0416a3c0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00011ad0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa04158da0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.411+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0416cbb0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.415+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ffa0001b3b0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.415+0000 7ff9fa7fc700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 0x7ff9e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.415+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6953+0+0 (secure 0 0 0) 0x7ffa0007bfd0 con 0x7ffa0406a4a0 2023-10-06T11:22:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.415+0000 7ffa098c0700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 0x7ff9e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.415+0000 7ffa098c0700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 0x7ff9e8062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa04064240 tx=0x7ff9f4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.423+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffa0004c040 con 0x7ffa0406a4a0 2023-10-06T11:22:51.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:51.623+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffa040cadd0 con 0x7ffa0406a4a0 2023-10-06T11:22:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.283+0000 7ff9fa7fc700 1 -- 172.21.15.7:0/848258339 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa040cadd0 con 0x7ffa0406a4a0 2023-10-06T11:22:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 msgr2=0x7ff9e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 0x7ff9e8062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa04064240 tx=0x7ff9f4017000 comp rx=0 tx=0).stop 2023-10-06T11:22:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 msgr2=0x7ffa04146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c590 tx=0x7ffa0000c8a0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 shutdown_connections 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff9e8060550 0x7ff9e8062a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffa040ca890 0x7ffa040b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa040c8970 0x7ffa04169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 --2- 172.21.15.7:0/848258339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa0406a4a0 0x7ffa04146990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 >> 172.21.15.7:0/848258339 conn(0x7ffa04067e30 msgr2=0x7ffa0415ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 shutdown_connections 2023-10-06T11:22:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.287+0000 7ffa0b0c3700 1 -- 172.21.15.7:0/848258339 wait complete. 2023-10-06T11:22:52.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:22:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.467+0000 7fe6e2733700 1 Processor -- start 2023-10-06T11:22:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.467+0000 7fe6e2733700 1 -- start start 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6dc06d940 0x7fe6dc06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe6dc160b80 0x7fe6dc162f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6dc0cc530 con 0x7fe6dc156b50 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6dc06e300 con 0x7fe6dc06d940 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6dc0cb090 con 0x7fe6dc160b80 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6dc06d940 0x7fe6dc06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6dc06d940 0x7fe6dc06ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57974/0 (socket says 172.21.15.7:57974) 2023-10-06T11:22:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 -- 172.21.15.7:0/985195551 learned_addr learned my addr 172.21.15.7:0/985195551 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e0cd0700 1 --2- 172.21.15.7:0/985195551 >> v2:172.21.15.50:3300/0 conn(0x7fe6dc160b80 0x7fe6dc162f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 -- 172.21.15.7:0/985195551 >> v2:172.21.15.50:3300/0 conn(0x7fe6dc160b80 msgr2=0x7fe6dc162f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/985195551 >> v2:172.21.15.50:3300/0 conn(0x7fe6dc160b80 0x7fe6dc162f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 -- 172.21.15.7:0/985195551 >> v1:172.21.15.50:6789/0 conn(0x7fe6dc156b50 legacy=0x7fe6dc159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 -- 172.21.15.7:0/985195551 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc15a090 con 0x7fe6dc06d940 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/985195551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc06d940 0x7fe6dc06ddc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ce020 tx=0x7fe6d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1046ee590078c313 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6daffd700 1 -- 172.21.15.7:0/985195551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0009e70 con 0x7fe6dc06d940 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6daffd700 1 -- 172.21.15.7:0/985195551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d000b860 con 0x7fe6dc06d940 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6daffd700 1 -- 172.21.15.7:0/985195551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000bc00 con 0x7fe6dc06d940 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/985195551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc06d940 msgr2=0x7fe6dc06ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 --2- 172.21.15.7:0/985195551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc06d940 0x7fe6dc06ddc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ce020 tx=0x7fe6d0009250 comp rx=0 tx=0).stop 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/985195551 shutdown_connections 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 --2- 172.21.15.7:0/985195551 >> v2:172.21.15.50:3300/0 conn(0x7fe6dc160b80 0x7fe6dc162f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 --2- 172.21.15.7:0/985195551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc06d940 0x7fe6dc06ddc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/985195551 >> 172.21.15.7:0/985195551 conn(0x7fe6dc0688c0 msgr2=0x7fe6dc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/985195551 shutdown_connections 2023-10-06T11:22:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/985195551 wait complete. 2023-10-06T11:22:52.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 Processor -- start 2023-10-06T11:22:52.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- start start 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe6dc160b80 0x7fe6dc0c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 0x7fe6dc1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc148710 con 0x7fe6dc156b50 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc15b0b0 con 0x7fe6dc0c5120 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc0c56e0 con 0x7fe6dc160b80 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57978/0 (socket says 172.21.15.7:57978) 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 -- 172.21.15.7:0/2587366385 learned_addr learned my addr 172.21.15.7:0/2587366385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe6dc160b80 0x7fe6dc0c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e0cd0700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 0x7fe6dc1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 -- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe6dc160b80 msgr2=0x7fe6dc0c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe6dc160b80 0x7fe6dc0c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 -- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 msgr2=0x7fe6dc1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 0x7fe6dc1581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 -- 172.21.15.7:0/2587366385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00094f0 con 0x7fe6dc156b50 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e0cd0700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 0x7fe6dc1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c590 tx=0x7fe6cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011590 con 0x7fe6dc156b50 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc011710 con 0x7fe6dc156b50 2023-10-06T11:22:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc0c59c0 con 0x7fe6dc156b50 2023-10-06T11:22:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011ad0 con 0x7fe6dc156b50 2023-10-06T11:22:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc0c5f80 con 0x7fe6dc156b50 2023-10-06T11:22:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.479+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc15ae80 con 0x7fe6dc156b50 2023-10-06T11:22:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.479+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe6cc01b3b0 con 0x7fe6dc156b50 2023-10-06T11:22:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.479+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 0x7fe6c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.479+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6953+0+0 (secure 0 0 0) 0x7fe6cc07c1f0 con 0x7fe6dc156b50 2023-10-06T11:22:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.479+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 0x7fe6c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.483+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 0x7fe6c8062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ce020 tx=0x7fe6d0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.483+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe6dc15ae80 con 0x7fe6dc156b50 2023-10-06T11:22:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:52.651+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6dc15ae80 con 0x7fe6dc156b50 2023-10-06T11:22:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.463+0000 7fe6c3fff700 1 -- 172.21.15.7:0/2587366385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe6dc15ae80 con 0x7fe6dc156b50 2023-10-06T11:22:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 msgr2=0x7fe6c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.471+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 0x7fe6c8062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ce020 tx=0x7fe6d0017000 comp rx=0 tx=0).stop 2023-10-06T11:22:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.471+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 msgr2=0x7fe6dc146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.471+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c590 tx=0x7fe6cc00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 shutdown_connections 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe6c8060550 0x7fe6c8062a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6dc0c5120 0x7fe6dc1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe6dc160b80 0x7fe6dc0c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 --2- 172.21.15.7:0/2587366385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc156b50 0x7fe6dc146ae0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 >> 172.21.15.7:0/2587366385 conn(0x7fe6dc0688c0 msgr2=0x7fe6dc14a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 shutdown_connections 2023-10-06T11:22:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.475+0000 7fe6e2733700 1 -- 172.21.15.7:0/2587366385 wait complete. 2023-10-06T11:22:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:22:53.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:22:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 Processor -- start 2023-10-06T11:22:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 -- start start 2023-10-06T11:22:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40158ea0 0x7f4c400ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c400ccd70 0x7f4c400b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c400cac70 con 0x7f4c4006d940 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c40068d40 con 0x7f4c40158ea0 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c450d3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400caea0 con 0x7f4c400ccd70 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40158ea0 0x7f4c400ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40158ea0 0x7f4c400ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57980/0 (socket says 172.21.15.7:57980) 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 -- 172.21.15.7:0/1707371517 learned_addr learned my addr 172.21.15.7:0/1707371517 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3f7fe700 1 --1- 172.21.15.7:0/1707371517 >> v1:172.21.15.7:6789/0 conn(0x7f4c4006d940 0x7f4c400cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44546/0 (socket says 172.21.15.7:44546) 2023-10-06T11:22:53.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c448d2700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c400ccd70 0x7f4c400b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 -- 172.21.15.7:0/1707371517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c400ccd70 msgr2=0x7f4c400b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c400ccd70 0x7f4c400b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 -- 172.21.15.7:0/1707371517 >> v1:172.21.15.7:6789/0 conn(0x7f4c4006d940 legacy=0x7f4c400cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3effd700 1 -- 172.21.15.7:0/1707371517 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021602350 0 0) 0x7f4c400cac70 con 0x7f4c4006d940 2023-10-06T11:22:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 -- 172.21.15.7:0/1707371517 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400cb180 con 0x7f4c40158ea0 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3ffff700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40158ea0 0x7f4c400ca730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4c280099b0 tx=0x7f4c28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e78df4dc9630a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3effd700 1 -- 172.21.15.7:0/1707371517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28009ea0 con 0x7f4c40158ea0 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3effd700 1 -- 172.21.15.7:0/1707371517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2800b860 con 0x7f4c40158ea0 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.599+0000 7f4c3effd700 1 -- 172.21.15.7:0/1707371517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800bc00 con 0x7f4c40158ea0 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- 172.21.15.7:0/1707371517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40158ea0 msgr2=0x7f4c400ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40158ea0 0x7f4c400ca730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4c280099b0 tx=0x7f4c28009250 comp rx=0 tx=0).stop 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- 172.21.15.7:0/1707371517 shutdown_connections 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c400ccd70 0x7f4c400b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 --2- 172.21.15.7:0/1707371517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40158ea0 0x7f4c400ca730 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- 172.21.15.7:0/1707371517 >> 172.21.15.7:0/1707371517 conn(0x7f4c40067e30 msgr2=0x7f4c40068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- 172.21.15.7:0/1707371517 shutdown_connections 2023-10-06T11:22:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- 172.21.15.7:0/1707371517 wait complete. 2023-10-06T11:22:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 Processor -- start 2023-10-06T11:22:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.603+0000 7f4c450d3700 1 -- start start 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c4006d940 0x7f4c400a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c40158ea0 0x7f4c40169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400b46d0 con 0x7f4c400ccd70 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400a8e00 con 0x7f4c40158ea0 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400ce1a0 con 0x7f4c4006d940 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3ffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c4006d940 0x7f4c400a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c448d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c40158ea0 0x7f4c40169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57982/0 (socket says 172.21.15.7:57982) 2023-10-06T11:22:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 -- 172.21.15.7:0/3598700953 learned_addr learned my addr 172.21.15.7:0/3598700953 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 -- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c4006d940 msgr2=0x7f4c400a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c4006d940 0x7f4c400a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 -- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c40158ea0 msgr2=0x7f4c40169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c40158ea0 0x7f4c40169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 -- 172.21.15.7:0/3598700953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c280094f0 con 0x7f4c400ccd70 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c3f7fe700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c5b0 tx=0x7f4c30001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30005a70 con 0x7f4c400ccd70 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c30005bd0 con 0x7f4c400ccd70 2023-10-06T11:22:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c400cf7c0 con 0x7f4c400ccd70 2023-10-06T11:22:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300070d0 con 0x7f4c400ccd70 2023-10-06T11:22:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.607+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4014d800 con 0x7f4c400ccd70 2023-10-06T11:22:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c400b48e0 con 0x7f4c400ccd70 2023-10-06T11:22:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4c30007480 con 0x7f4c400ccd70 2023-10-06T11:22:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c37fff700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 0x7f4c20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6515+0+0 (secure 0 0 0) 0x7f4c3007bfe0 con 0x7f4c400ccd70 2023-10-06T11:22:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c3ffff700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 0x7f4c20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.611+0000 7f4c3ffff700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 0x7f4c20062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c280099b0 tx=0x7f4c28017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.615+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c400b48e0 con 0x7f4c400ccd70 2023-10-06T11:22:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:53.787+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c400b48e0 con 0x7f4c400ccd70 2023-10-06T11:22:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.479+0000 7f4c37fff700 1 -- 172.21.15.7:0/3598700953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4c400b48e0 con 0x7f4c400ccd70 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 msgr2=0x7f4c20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 0x7f4c20062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c280099b0 tx=0x7f4c28017000 comp rx=0 tx=0).stop 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 msgr2=0x7f4c4014d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c5b0 tx=0x7f4c30001040 comp rx=0 tx=0).stop 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 shutdown_connections 2023-10-06T11:22:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4c20060550 0x7f4c20062a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c40158ea0 0x7f4c40169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c400ccd70 0x7f4c4014d2c0 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c5b0 tx=0x7f4c30001040 comp rx=0 tx=0).stop 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 --2- 172.21.15.7:0/3598700953 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4c4006d940 0x7f4c400a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 >> 172.21.15.7:0/3598700953 conn(0x7f4c40067e30 msgr2=0x7f4c400c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.487+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 shutdown_connections 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.491+0000 7f4c450d3700 1 -- 172.21.15.7:0/3598700953 wait complete. 2023-10-06T11:22:54.494 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-10-06T11:22:54.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-06T11:22:54.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 Processor -- start 2023-10-06T11:22:54.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 -- start start 2023-10-06T11:22:54.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f07ac0cc1a0 0x7f07ac156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0cadf0 con 0x7f07ac0b10b0 2023-10-06T11:22:54.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0cb280 con 0x7f07ac1568b0 2023-10-06T11:22:54.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2e31700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0c8370 con 0x7f07ac0cc1a0 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b162e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07ac0b10b0 0x7f07ac15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44550/0 (socket says 172.21.15.7:44550) 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b162e700 1 -- 172.21.15.7:0/3541764180 learned_addr learned my addr 172.21.15.7:0/3541764180 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2630700 1 --2- 172.21.15.7:0/3541764180 >> v2:172.21.15.50:3300/0 conn(0x7f07ac0cc1a0 0x7f07ac156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058825401 0 0) 0x7f07ac0cadf0 con 0x7f07ac0b10b0 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0798003570 con 0x7f07ac0b10b0 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348669591 0 0) 0x7f0798003570 con 0x7f07ac0b10b0 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 >> v2:172.21.15.50:3300/0 conn(0x7f07ac0cc1a0 msgr2=0x7f07ac156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 --2- 172.21.15.7:0/3541764180 >> v2:172.21.15.50:3300/0 conn(0x7f07ac0cc1a0 0x7f07ac156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 >> v1:172.21.15.50:6789/0 conn(0x7f07ac1568b0 legacy=0x7f07ac0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0c8630 con 0x7f07ac0b10b0 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f079c001470 con 0x7f07ac0b10b0 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 >> v1:172.21.15.7:6789/0 conn(0x7f07ac0b10b0 legacy=0x7f07ac15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 0x7f0798003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0798004810 0x7f0798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f07980051c0 0x7f079800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0c8370 con 0x7f0798003a60 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079800c160 con 0x7f0798004810 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079800c320 con 0x7f07980051c0 2023-10-06T11:22:54.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 0x7f0798003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b162e700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0798004810 0x7f0798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2630700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f07980051c0 0x7f079800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 -- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f07980051c0 msgr2=0x7f079800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f07980051c0 0x7f079800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 -- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0798004810 msgr2=0x7f0798004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0798004810 0x7f0798004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 -- 172.21.15.7:0/3541764180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798003570 con 0x7f0798003a60 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b2630700 1 -- 172.21.15.7:0/3541764180 reap_dead start 2023-10-06T11:22:54.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 0x7f0798003e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a80018b0 tx=0x7f07a8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c75f94f9b2ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:54.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8012420 con 0x7f0798003a60 2023-10-06T11:22:54.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80125a0 con 0x7f0798003a60 2023-10-06T11:22:54.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.623+0000 7f07b0e2d700 1 -- 172.21.15.7:0/3541764180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8012960 con 0x7f0798003a60 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 -- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 msgr2=0x7f0798003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 0x7f0798003e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a80018b0 tx=0x7f07a8007d30 comp rx=0 tx=0).stop 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 -- 172.21.15.7:0/3541764180 shutdown_connections 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 --2- 172.21.15.7:0/3541764180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0798003a60 0x7f0798003e70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 -- 172.21.15.7:0/3541764180 >> 172.21.15.7:0/3541764180 conn(0x7f07ac068880 msgr2=0x7f07ac0cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:54.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 -- 172.21.15.7:0/3541764180 shutdown_connections 2023-10-06T11:22:54.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 -- 172.21.15.7:0/3541764180 wait complete. 2023-10-06T11:22:54.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.627+0000 7f07b2e31700 1 Processor -- start 2023-10-06T11:22:54.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 -- start start 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0798003a60 0x7f07ac14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac14fef0 0x7f07ac16a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac166d40 con 0x7f07ac150430 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac15b390 con 0x7f07ac14fef0 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac15b670 con 0x7f0798003a60 2023-10-06T11:22:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57996/0 (socket says 172.21.15.7:57996) 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 -- 172.21.15.7:0/693390440 learned_addr learned my addr 172.21.15.7:0/693390440 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0798003a60 0x7f07ac14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b162e700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac14fef0 0x7f07ac16a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 -- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0798003a60 msgr2=0x7f07ac14f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0798003a60 0x7f07ac14f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 -- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac14fef0 msgr2=0x7f07ac16a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac14fef0 0x7f07ac16a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 -- 172.21.15.7:0/693390440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac15b950 con 0x7f07ac150430 2023-10-06T11:22:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2630700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0cad60 tx=0x7f07a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400e070 con 0x7f07ac150430 2023-10-06T11:22:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4011b80 con 0x7f07ac150430 2023-10-06T11:22:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac15bc30 con 0x7f07ac150430 2023-10-06T11:22:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.631+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4016470 con 0x7f07ac150430 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac15b020 con 0x7f07ac150430 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f07a4016860 con 0x7f07ac150430 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac15be70 con 0x7f07ac150430 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07a1ffb700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 0x7f0790062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6043+0+0 (secure 0 0 0) 0x7f07a407d350 con 0x7f07ac150430 2023-10-06T11:22:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 0x7f0790062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.635+0000 7f07b1e2f700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 0x7f0790062a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07ac155590 tx=0x7f07a8015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.643+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f07a404d7f0 con 0x7f07ac150430 2023-10-06T11:22:54.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:54.803+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07ac0c8910 con 0x7f07ac150430 2023-10-06T11:22:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.495+0000 7f07a1ffb700 1 -- 172.21.15.7:0/693390440 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f07a4050e10 con 0x7f07ac150430 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 msgr2=0x7f0790062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 0x7f0790062a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07ac155590 tx=0x7f07a8015000 comp rx=0 tx=0).stop 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 msgr2=0x7f07ac153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0cad60 tx=0x7f07a400b040 comp rx=0 tx=0).stop 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 shutdown_connections 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0790060580 0x7f0790062a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07ac150430 0x7f07ac153140 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac14fef0 0x7f07ac16a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 --2- 172.21.15.7:0/693390440 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0798003a60 0x7f07ac14f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 >> 172.21.15.7:0/693390440 conn(0x7f07ac068880 msgr2=0x7f07ac069300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 shutdown_connections 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.499+0000 7f07b2e31700 1 -- 172.21.15.7:0/693390440 wait complete. 2023-10-06T11:22:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2023-10-06T11:22:55.518 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:22:55.518 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:22:55.518 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:22:55.518 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:22:55.519 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:22:55.519 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:22:55.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:22:55.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:22:55.519 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:22:55.520 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:22:55.520 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:22:55.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:22:55.520 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:22:55.520 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:22:55.520 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:22:55.521 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:22:55.530 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:22:55.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:22:55.531 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:22:55.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:22:55.532 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:22:55.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:22:55.534 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:22:55.534 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:22:55.535 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:22:55.535 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:55.536 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:22:55.536 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:55.537 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:22:55.537 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:55.537 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:22:55.538 DEBUG:teuthology.parallel:result is None 2023-10-06T11:22:55.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:22:55.607 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-06T11:22:55.607 INFO:tasks.ceph.mds.b.smithi050.stdout:starting mds.b at 2023-10-06T11:22:55.611 INFO:tasks.ceph.mds.d.smithi050.stdout:starting mds.d at 2023-10-06T11:22:55.620 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-06T11:22:55.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 Processor -- start 2023-10-06T11:22:55.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 -- start start 2023-10-06T11:22:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6ca415cb40 0x7f6ca40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca4069eb0 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca40c8770 con 0x7f6ca415cb40 2023-10-06T11:22:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca9a3c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c65c0 con 0x7f6ca40addf0 2023-10-06T11:22:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca27fc700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6ca415cb40 0x7f6ca40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51042/0 (socket says 172.21.15.7:51042) 2023-10-06T11:22:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/1617627858 learned_addr learned my addr 172.21.15.7:0/1617627858 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/1617627858 >> v2:172.21.15.50:3300/0 conn(0x7f6ca415cb40 msgr2=0x7f6ca40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/1617627858 >> v2:172.21.15.50:3300/0 conn(0x7f6ca415cb40 0x7f6ca40c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca1ffb700 1 -- 172.21.15.7:0/1617627858 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724604017 0 0) 0x7f6ca4069eb0 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/1617627858 >> v1:172.21.15.50:6789/0 conn(0x7f6ca40ae1c0 legacy=0x7f6ca406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/1617627858 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca415a4b0 con 0x7f6ca40addf0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.659+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/1617627858 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f20 tx=0x7f6c9c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4baee2fe6217a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca1ffb700 1 -- 172.21.15.7:0/1617627858 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00e070 con 0x7f6ca40addf0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca1ffb700 1 -- 172.21.15.7:0/1617627858 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00b9d0 con 0x7f6ca40addf0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca1ffb700 1 -- 172.21.15.7:0/1617627858 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0092e0 con 0x7f6ca40addf0 2023-10-06T11:22:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/1617627858 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 msgr2=0x7f6ca4159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/1617627858 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f20 tx=0x7f6c9c009840 comp rx=0 tx=0).stop 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/1617627858 shutdown_connections 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/1617627858 >> v2:172.21.15.50:3300/0 conn(0x7f6ca415cb40 0x7f6ca40c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/1617627858 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca4159f70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/1617627858 >> 172.21.15.7:0/1617627858 conn(0x7f6ca4067e30 msgr2=0x7f6ca4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/1617627858 shutdown_connections 2023-10-06T11:22:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.663+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/1617627858 wait complete. 2023-10-06T11:22:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 Processor -- start 2023-10-06T11:22:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- start start 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca415cb40 0x7f6ca4158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca414dc70 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40af4f0 con 0x7f6ca415cb40 2023-10-06T11:22:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40af7d0 con 0x7f6ca40addf0 2023-10-06T11:22:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58046/0 (socket says 172.21.15.7:58046) 2023-10-06T11:22:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca2ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca415cb40 0x7f6ca4158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 -- 172.21.15.7:0/2271446825 learned_addr learned my addr 172.21.15.7:0/2271446825 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 -- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 msgr2=0x7f6ca414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca414d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 -- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca415cb40 msgr2=0x7f6ca4158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca415cb40 0x7f6ca4158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 -- 172.21.15.7:0/2271446825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009ae0 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca27fc700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00e9d0 tx=0x7f6c8c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c013680 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0137e0 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40afab0 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c013b80 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.667+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca40b0070 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.671+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4070f20 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.671+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6c8c00c490 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.671+0000 7f6c9affd700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 0x7f6c84062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.671+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f6c8c07cd90 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.671+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 0x7f6c84062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.675+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 0x7f6c84062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009880 tx=0x7f6c9c018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:55.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.679+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6c8c04d340 con 0x7f6ca40ae1c0 2023-10-06T11:22:55.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:55.855+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6ca415d040 con 0x7f6ca40ae1c0 2023-10-06T11:22:56.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.511+0000 7f6c9affd700 1 -- 172.21.15.7:0/2271446825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c8c050960 con 0x7f6ca40ae1c0 2023-10-06T11:22:56.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 msgr2=0x7f6c84062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 0x7f6c84062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009880 tx=0x7f6c9c018000 comp rx=0 tx=0).stop 2023-10-06T11:22:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 msgr2=0x7f6ca41584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00e9d0 tx=0x7f6c8c009120 comp rx=0 tx=0).stop 2023-10-06T11:22:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 shutdown_connections 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6c84060550 0x7f6c84062a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca415cb40 0x7f6ca4158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40ae1c0 0x7f6ca41584f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 --2- 172.21.15.7:0/2271446825 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f6ca40addf0 0x7f6ca414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.519+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 >> 172.21.15.7:0/2271446825 conn(0x7f6ca4067e30 msgr2=0x7f6ca414e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.523+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 shutdown_connections 2023-10-06T11:22:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.523+0000 7f6ca9a3c700 1 -- 172.21.15.7:0/2271446825 wait complete. 2023-10-06T11:22:56.527 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2023-10-06T11:22:56.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:22:56.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.635+0000 7f348a27c700 1 Processor -- start 2023-10-06T11:22:56.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348a27c700 1 -- start start 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348a27c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f348406eb80 0x7f348406bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348a27c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348406e6f0 con 0x7f348406bad0 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348927a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f348406bad0 0x7f34840cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44560/0 (socket says 172.21.15.7:44560) 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348927a700 1 -- 172.21.15.7:0/3589136454 learned_addr learned my addr 172.21.15.7:0/3589136454 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840cbc70 con 0x7f3484155850 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840cbf50 con 0x7f348406eb80 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769624099 0 0) 0x7f348406e6f0 con 0x7f348406bad0 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3470003570 con 0x7f348406bad0 2023-10-06T11:22:56.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f3488a79700 1 --2- 172.21.15.7:0/3589136454 >> v2:172.21.15.50:3300/0 conn(0x7f348406eb80 0x7f348406bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3879625454 0 0) 0x7f3470003570 con 0x7f348406bad0 2023-10-06T11:22:56.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 >> v2:172.21.15.50:3300/0 conn(0x7f348406eb80 msgr2=0x7f348406bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 --2- 172.21.15.7:0/3589136454 >> v2:172.21.15.50:3300/0 conn(0x7f348406eb80 0x7f348406bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 >> v1:172.21.15.50:6789/0 conn(0x7f3484155850 legacy=0x7f3484155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348406e0a0 con 0x7f348406bad0 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f3480001470 con 0x7f348406bad0 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 >> v1:172.21.15.7:6789/0 conn(0x7f348406bad0 legacy=0x7f34840cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 0x7f3470003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470004810 0x7f3470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34700052e0 0x7f347000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cbf50 con 0x7f3470003a60 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347000c190 con 0x7f34700052e0 2023-10-06T11:22:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347000c350 con 0x7f3470004810 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f348927a700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 0x7f3470003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f3489a7b700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34700052e0 0x7f347000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.639+0000 7f3488a79700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470004810 0x7f3470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 -- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470004810 msgr2=0x7f3470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470004810 0x7f3470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 -- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34700052e0 msgr2=0x7f347000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34700052e0 0x7f347000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 -- 172.21.15.7:0/3589136454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470003570 con 0x7f3470003a60 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f3489a7b700 1 -- 172.21.15.7:0/3589136454 reap_dead start 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348927a700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 0x7f3470003e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f348000c3c0 tx=0x7f3480001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5192f42bc76e72fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000cc90 con 0x7f3470003a60 2023-10-06T11:22:56.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348000cdf0 con 0x7f3470003a60 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f347bfff700 1 -- 172.21.15.7:0/3589136454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480005b50 con 0x7f3470003a60 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 msgr2=0x7f3470003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 0x7f3470003e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f348000c3c0 tx=0x7f3480001e50 comp rx=0 tx=0).stop 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 shutdown_connections 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 --2- 172.21.15.7:0/3589136454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470003a60 0x7f3470003e70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 >> 172.21.15.7:0/3589136454 conn(0x7f34840cc190 msgr2=0x7f34840cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 shutdown_connections 2023-10-06T11:22:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- 172.21.15.7:0/3589136454 wait complete. 2023-10-06T11:22:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 Processor -- start 2023-10-06T11:22:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.643+0000 7f348a27c700 1 -- start start 2023-10-06T11:22:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470003a60 0x7f348414df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 0x7f3484157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348406aae0 con 0x7f3484069cd0 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348416a9a0 con 0x7f348416b370 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348406a240 con 0x7f3470003a60 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58066/0 (socket says 172.21.15.7:58066) 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 -- 172.21.15.7:0/891099446 learned_addr learned my addr 172.21.15.7:0/891099446 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348927a700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470003a60 0x7f348414df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3489a7b700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 0x7f3484157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 -- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470003a60 msgr2=0x7f348414df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470003a60 0x7f348414df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 -- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 msgr2=0x7f3484157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 0x7f3484157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 -- 172.21.15.7:0/891099446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348406a520 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3489a7b700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 0x7f3484157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f3488a79700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34740109d0 tx=0x7f3474008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400e870 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400e9d0 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348406a800 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400ed90 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840c23d0 con 0x7f3484069cd0 2023-10-06T11:22:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348406d240 con 0x7f3484069cd0 2023-10-06T11:22:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.647+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3474014640 con 0x7f3484069cd0 2023-10-06T11:22:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.651+0000 7f34797fa700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 0x7f3468062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.651+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f347407cd30 con 0x7f3484069cd0 2023-10-06T11:22:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.651+0000 7f348927a700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 0x7f3468062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.651+0000 7f348927a700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 0x7f3468062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.655+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f347404d3d0 con 0x7f3484069cd0 2023-10-06T11:22:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f348416aae0 con 0x7f3484069cd0 2023-10-06T11:22:56.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f34797fa700 1 -- 172.21.15.7:0/891099446 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3474018090 con 0x7f3484069cd0 2023-10-06T11:22:56.879 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:56.879 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:22:56.879 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2023-10-06T11:22:56.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9l1+wnCxAAWCyrtRm4/l01bVOfl3RFNA==", 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.880 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9laHI5FRAA3af4MNcnGb9VDo5Ijo30SA==", 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lXLcqERAANaP6kbnJHMe9OCl2sjMa1g==", 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lVyWRGBAA5iEiW4W9odLEUorf3t5EPQ==", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2023-10-06T11:22:56.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCj7R9lGH/WCxAAzuwYCO4wsYR2zGf1W29XZw==", 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2023-10-06T11:22:56.884 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCm7R9lY1MzEBAAlxX4RI/KFowzyFqUncP6xQ==", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCp7R9lJykPFxAAxwaaC+xoN9U+SHpI2iz1/Q==", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCs7R9l1vLTHBAAxImlU3qarF5ZOKn+wiKYMA==", 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCw7R9lVEbJBRAAj48N2fdwZO74MPfqZrBgLQ==", 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCz7R9l0YT3BxAAh/FAxhszZOD8HHADQWv86Q==", 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2023-10-06T11:22:56.888 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC27R9lwmb3CRAA7kBK8youRc70G4y9C7DZnA==", 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC57R9l9PngDxAAiYqjQtUm9kSgphe0jP8Jeg==", 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lAsmfGxAA/rcxLJ7QHrdvJ+qQEx6gOw==", 2023-10-06T11:22:56.890 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2023-10-06T11:22:56.891 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lip1bIRAAuhH8cb2ryJKFjGCWT7lUVg==", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lAXRWHhAAzypkRTEsOQwk3ogrI6HrMQ==", 2023-10-06T11:22:56.892 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lov88JBAAR9JDw9oMtv3OKzAZ5F+kiw==", 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCh7R9ldPsZNBAAlnqx1zhoG+qehWBUQt+StA==", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-06T11:22:56.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9l0ge1HBAApATPXZVsl6k0s7oz1CVZtw==", 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9lWx21HBAAM7+x997Kx1EYFbTlB2cH6Q==", 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9lfDG1HBAA/Z+Zzm/MCxDf21vscQMh+Q==", 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9ln0K1HBAAUlI7LWzYYdsY9IH2MlUf/Q==", 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9lj1O1HBAAJFbn9tCOC3YLMtz1R58vBw==", 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDD7R9lD2W1HBAA6f8e7c3X8hUNB/9wnI0ksw==", 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T11:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9lXbVmCBAAPUux0zkxVGM3gHdQhuk4VQ==", 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCi7R9ldAdbBRAA8GCg4bKcrzuzkaduCi8XvA==", 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-06T11:22:56.900 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:22:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 msgr2=0x7f3468062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 0x7f3468062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480015000 comp rx=0 tx=0).stop 2023-10-06T11:22:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 msgr2=0x7f348416ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.875+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34740109d0 tx=0x7f3474008040 comp rx=0 tx=0).stop 2023-10-06T11:22:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 shutdown_connections 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3468060550 0x7f3468062a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348416b370 0x7f3484157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3484069cd0 0x7f348416ae30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 --2- 172.21.15.7:0/891099446 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3470003a60 0x7f348414df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 >> 172.21.15.7:0/891099446 conn(0x7f34840cc190 msgr2=0x7f348415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 shutdown_connections 2023-10-06T11:22:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:56.879+0000 7f348a27c700 1 -- 172.21.15.7:0/891099446 wait complete. 2023-10-06T11:22:56.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T11:22:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 Processor -- start 2023-10-06T11:22:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 -- start start 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f668c0b1150 0x7f668c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0cabe0 con 0x7f668c158660 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0caec0 con 0x7f668c1569d0 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6692764700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c0cb1a0 con 0x7f668c0b1150 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6690f61700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f668c158660 0x7f668c0b0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44568/0 (socket says 172.21.15.7:44568) 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6690f61700 1 -- 172.21.15.7:0/3130237672 learned_addr learned my addr 172.21.15.7:0/3130237672 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.011+0000 7f6691f63700 1 --2- 172.21.15.7:0/3130237672 >> v2:172.21.15.50:3300/0 conn(0x7f668c0b1150 0x7f668c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323238444 0 0) 0x7f668c0cabe0 con 0x7f668c158660 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6678003570 con 0x7f668c158660 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2891955526 0 0) 0x7f6678003570 con 0x7f668c158660 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 >> v2:172.21.15.50:3300/0 conn(0x7f668c0b1150 msgr2=0x7f668c156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 --2- 172.21.15.7:0/3130237672 >> v2:172.21.15.50:3300/0 conn(0x7f668c0b1150 0x7f668c156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 >> v1:172.21.15.50:6789/0 conn(0x7f668c1569d0 legacy=0x7f668c0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0c8ad0 con 0x7f668c158660 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f667c001490 con 0x7f668c158660 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 >> v1:172.21.15.7:6789/0 conn(0x7f668c158660 legacy=0x7f668c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f6678003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678004810 0x7f6678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f66780051c0 0x7f667800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0cb1a0 con 0x7f6678004810 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667800c160 con 0x7f6678003a60 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667800c320 con 0x7f66780051c0 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6690f61700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678004810 0x7f6678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f6678003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691f63700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f66780051c0 0x7f667800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 -- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f66780051c0 msgr2=0x7f667800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f66780051c0 0x7f667800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 -- 172.21.15.7:0/3130237672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678004810 msgr2=0x7f6678004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678004810 0x7f6678004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 -- 172.21.15.7:0/3130237672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678003570 con 0x7f6678003a60 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6690f61700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678004810 0x7f6678004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691f63700 1 -- 172.21.15.7:0/3130237672 reap_dead start 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6691762700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f6678003e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6688001fc0 tx=0x7f6688001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca4a4ecc7fdb503 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688001950 con 0x7f6678003a60 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800c8b0 con 0x7f6678003a60 2023-10-06T11:22:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6683fff700 1 -- 172.21.15.7:0/3130237672 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800cc50 con 0x7f6678003a60 2023-10-06T11:22:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 -- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 msgr2=0x7f6678003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f6678003e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6688001fc0 tx=0x7f6688001640 comp rx=0 tx=0).stop 2023-10-06T11:22:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 -- 172.21.15.7:0/3130237672 shutdown_connections 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 --2- 172.21.15.7:0/3130237672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f6678003e70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 -- 172.21.15.7:0/3130237672 >> 172.21.15.7:0/3130237672 conn(0x7f668c0688a0 msgr2=0x7f668c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 -- 172.21.15.7:0/3130237672 shutdown_connections 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.015+0000 7f6692764700 1 -- 172.21.15.7:0/3130237672 wait complete. 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 Processor -- start 2023-10-06T11:22:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 -- start start 2023-10-06T11:22:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f668c14db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 0x7f668c16a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f668c16a620 0x7f668c16aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c167a90 con 0x7f668c14e080 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c14f3a0 con 0x7f6678003a60 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6692764700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c14e4f0 con 0x7f668c16a620 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6691762700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f668c14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6690f61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 0x7f668c16a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6691f63700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f668c16a620 0x7f668c16aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6691762700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f668c14db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:36008/0 (socket says 172.21.15.7:36008) 2023-10-06T11:22:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.019+0000 7f6691762700 1 -- 172.21.15.7:0/1355764141 learned_addr learned my addr 172.21.15.7:0/1355764141 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 -- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f668c16a620 msgr2=0x7f668c16aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f668c16a620 0x7f668c16aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 -- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 msgr2=0x7f668c14db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f668c14db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 -- 172.21.15.7:0/1355764141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c14e790 con 0x7f668c14e080 2023-10-06T11:22:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6690f61700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 0x7f668c16a0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c018030 con 0x7f668c14e080 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c00f970 con 0x7f668c14e080 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00fcf0 con 0x7f668c14e080 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c14ea70 con 0x7f668c14e080 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c14f030 con 0x7f668c14e080 2023-10-06T11:22:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c14f550 con 0x7f668c14e080 2023-10-06T11:22:57.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.023+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f667c01b070 con 0x7f668c14e080 2023-10-06T11:22:57.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.027+0000 7f66817fa700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 0x7f6670062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.027+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f667c019070 con 0x7f668c14e080 2023-10-06T11:22:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.027+0000 7f6691762700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 0x7f6670062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.027+0000 7f6691762700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 0x7f6670062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6688001da0 tx=0x7f6688015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.031+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f668c14f550 con 0x7f668c14e080 2023-10-06T11:22:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.251+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f668c14f510 con 0x7f668c14e080 2023-10-06T11:22:57.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.259+0000 7f66817fa700 1 -- 172.21.15.7:0/1355764141 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f667c04cd40 con 0x7f668c14e080 2023-10-06T11:22:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.263+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 msgr2=0x7f6670062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.263+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 0x7f6670062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6688001da0 tx=0x7f6688015000 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.263+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 msgr2=0x7f668c16a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.263+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 0x7f668c16a0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c001e70 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.263+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 shutdown_connections 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6670060550 0x7f6670062a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f668c16a620 0x7f668c16aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f668c14e080 0x7f668c16a0e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 --2- 172.21.15.7:0/1355764141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6678003a60 0x7f668c14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 >> 172.21.15.7:0/1355764141 conn(0x7f668c0688a0 msgr2=0x7f668c0695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 shutdown_connections 2023-10-06T11:22:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.267+0000 7f6692764700 1 -- 172.21.15.7:0/1355764141 wait complete. 2023-10-06T11:22:57.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T11:22:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 Processor -- start 2023-10-06T11:22:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 -- start start 2023-10-06T11:22:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7be0066840 0x7f7be0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be0066d40 con 0x7f7be015a460 2023-10-06T11:22:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be0068750 con 0x7f7be0155a00 2023-10-06T11:22:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be7208700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be0068540 con 0x7f7be0066840 2023-10-06T11:22:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be6a07700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7be015a460 0x7f7be00c74e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44574/0 (socket says 172.21.15.7:44574) 2023-10-06T11:22:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be6a07700 1 -- 172.21.15.7:0/2903003325 learned_addr learned my addr 172.21.15.7:0/2903003325 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 --2- 172.21.15.7:0/2903003325 >> v2:172.21.15.50:3300/0 conn(0x7f7be0066840 0x7f7be0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457026806 0 0) 0x7f7be0066d40 con 0x7f7be015a460 2023-10-06T11:22:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bcc003570 con 0x7f7be015a460 2023-10-06T11:22:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755465874 0 0) 0x7f7bcc003570 con 0x7f7be015a460 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 >> v2:172.21.15.50:3300/0 conn(0x7f7be0066840 msgr2=0x7f7be0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 --2- 172.21.15.7:0/2903003325 >> v2:172.21.15.50:3300/0 conn(0x7f7be0066840 0x7f7be0159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 >> v1:172.21.15.50:6789/0 conn(0x7f7be0155a00 legacy=0x7f7be0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0068a30 con 0x7f7be015a460 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f7bd80017f0 con 0x7f7be015a460 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 >> v1:172.21.15.7:6789/0 conn(0x7f7be015a460 legacy=0x7f7be00c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bcc003a60 0x7f7bcc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bcc0051c0 0x7f7bcc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0068540 con 0x7f7bcc004810 2023-10-06T11:22:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc00e3f0 con 0x7f7bcc0051c0 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc00e5b0 con 0x7f7bcc003a60 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be6a07700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bcc0051c0 0x7f7bcc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be6206700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bcc003a60 0x7f7bcc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 -- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bcc003a60 msgr2=0x7f7bcc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7bcc003a60 0x7f7bcc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 -- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bcc0051c0 msgr2=0x7f7bcc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bcc0051c0 0x7f7bcc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 -- 172.21.15.7:0/2903003325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc003570 con 0x7f7bcc004810 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be6a07700 1 -- 172.21.15.7:0/2903003325 reap_dead start 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5a05700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7bcc004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006f00 tx=0x7f7bd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59e58e407462bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd000e6e0 con 0x7f7bcc004810 2023-10-06T11:22:57.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd000e840 con 0x7f7bcc004810 2023-10-06T11:22:57.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.391+0000 7f7be5204700 1 -- 172.21.15.7:0/2903003325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd000ec00 con 0x7f7bcc004810 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 msgr2=0x7f7bcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7bcc004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006f00 tx=0x7f7bd00091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- 172.21.15.7:0/2903003325 shutdown_connections 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 --2- 172.21.15.7:0/2903003325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7bcc004c80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- 172.21.15.7:0/2903003325 >> 172.21.15.7:0/2903003325 conn(0x7f7be006a990 msgr2=0x7f7be0067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- 172.21.15.7:0/2903003325 shutdown_connections 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- 172.21.15.7:0/2903003325 wait complete. 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 Processor -- start 2023-10-06T11:22:57.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 -- start start 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.395+0000 7f7be7208700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7be0155dd0 0x7f7be00c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 0x7f7be01533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0159340 con 0x7f7bcc004810 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00c1570 con 0x7f7be0156240 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00c0930 con 0x7f7be0155dd0 2023-10-06T11:22:57.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58108/0 (socket says 172.21.15.7:58108) 2023-10-06T11:22:57.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 -- 172.21.15.7:0/828494735 learned_addr learned my addr 172.21.15.7:0/828494735 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:57.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6a07700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 0x7f7be01533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be5a05700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7be0155dd0 0x7f7be00c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 -- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7be0155dd0 msgr2=0x7f7be00c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7be0155dd0 0x7f7be00c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 -- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 msgr2=0x7f7be01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 0x7f7be01533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 -- 172.21.15.7:0/828494735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00c0c10 con 0x7f7bcc004810 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6a07700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 0x7f7be01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be6206700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e890 tx=0x7f7bdc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc013630 con 0x7f7bcc004810 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc0137b0 con 0x7f7bcc004810 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be00c1200 con 0x7f7bcc004810 2023-10-06T11:22:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc013b30 con 0x7f7bcc004810 2023-10-06T11:22:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be00c05c0 con 0x7f7bcc004810 2023-10-06T11:22:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be00c1060 con 0x7f7bcc004810 2023-10-06T11:22:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.399+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bdc01c400 con 0x7f7bcc004810 2023-10-06T11:22:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.403+0000 7f7bd67fc700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 0x7f7bc4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.403+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7bdc07ccb0 con 0x7f7bcc004810 2023-10-06T11:22:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.403+0000 7f7be5a05700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 0x7f7bc4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.403+0000 7f7be5a05700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 0x7f7bc4062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00091e0 tx=0x7f7bd001a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.407+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7be00c1060 con 0x7f7bcc004810 2023-10-06T11:22:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.623+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f7be00c1060 con 0x7f7bcc004810 2023-10-06T11:22:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.635+0000 7f7bd67fc700 1 -- 172.21.15.7:0/828494735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7be00c1060 con 0x7f7bcc004810 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 msgr2=0x7f7bc4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 0x7f7bc4062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00091e0 tx=0x7f7bd001a000 comp rx=0 tx=0).stop 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 msgr2=0x7f7be014f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e890 tx=0x7f7bdc009040 comp rx=0 tx=0).stop 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 shutdown_connections 2023-10-06T11:22:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7bc4060550 0x7f7bc4062a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be0156240 0x7f7be01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7be0155dd0 0x7f7be00c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 --2- 172.21.15.7:0/828494735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bcc004810 0x7f7be014f830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 >> 172.21.15.7:0/828494735 conn(0x7f7be006a990 msgr2=0x7f7be0151690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 shutdown_connections 2023-10-06T11:22:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.639+0000 7f7be7208700 1 -- 172.21.15.7:0/828494735 wait complete. 2023-10-06T11:22:57.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T11:22:57.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.759+0000 7f5f36ad6700 1 Processor -- start 2023-10-06T11:22:57.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.759+0000 7f5f36ad6700 1 -- start start 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5f30158b40 0x7f5f300caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f300cb1e0 0x7f5f300cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3006e310 con 0x7f5f30158f10 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3006e7a0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3006d720 con 0x7f5f30158b40 2023-10-06T11:22:57.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f300cb1e0 0x7f5f300cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f300cb1e0 0x7f5f300cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58116/0 (socket says 172.21.15.7:58116) 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 -- 172.21.15.7:0/1283851169 learned_addr learned my addr 172.21.15.7:0/1283851169 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f35ad4700 1 --2- 172.21.15.7:0/1283851169 >> v2:172.21.15.50:3300/0 conn(0x7f5f30158b40 0x7f5f300caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 -- 172.21.15.7:0/1283851169 >> v2:172.21.15.50:3300/0 conn(0x7f5f30158b40 msgr2=0x7f5f300caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 --2- 172.21.15.7:0/1283851169 >> v2:172.21.15.50:3300/0 conn(0x7f5f30158b40 0x7f5f300caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 -- 172.21.15.7:0/1283851169 >> v1:172.21.15.50:6789/0 conn(0x7f5f30158f10 legacy=0x7f5f3015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 -- 172.21.15.7:0/1283851169 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3006d9e0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f352d3700 1 --2- 172.21.15.7:0/1283851169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f300cc550 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f300cd7f0 tx=0x7f5f2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e326f452184c50d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f34ad2700 1 -- 172.21.15.7:0/1283851169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00b860 con 0x7f5f300cb1e0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f34ad2700 1 -- 172.21.15.7:0/1283851169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c00b9c0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f34ad2700 1 -- 172.21.15.7:0/1283851169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00bd60 con 0x7f5f300cb1e0 2023-10-06T11:22:57.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- 172.21.15.7:0/1283851169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 msgr2=0x7f5f300cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/1283851169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f300cc550 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f300cd7f0 tx=0x7f5f2c009250 comp rx=0 tx=0).stop 2023-10-06T11:22:57.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- 172.21.15.7:0/1283851169 shutdown_connections 2023-10-06T11:22:57.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/1283851169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f300cc550 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/1283851169 >> v2:172.21.15.50:3300/0 conn(0x7f5f30158b40 0x7f5f300caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- 172.21.15.7:0/1283851169 >> 172.21.15.7:0/1283851169 conn(0x7f5f30067e30 msgr2=0x7f5f30068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- 172.21.15.7:0/1283851169 shutdown_connections 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.763+0000 7f5f36ad6700 1 -- 172.21.15.7:0/1283851169 wait complete. 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 Processor -- start 2023-10-06T11:22:57.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- start start 2023-10-06T11:22:57.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30158b40 0x7f5f30156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5f30158f10 0x7f5f300ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30157960 con 0x7f5f300cb1e0 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30156d50 con 0x7f5f30158b40 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30157030 con 0x7f5f30158f10 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58120/0 (socket says 172.21.15.7:58120) 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 -- 172.21.15.7:0/3584328712 learned_addr learned my addr 172.21.15.7:0/3584328712 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:57.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f352d3700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30158b40 0x7f5f30156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f362d5700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5f30158f10 0x7f5f300ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 -- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5f30158f10 msgr2=0x7f5f300ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5f30158f10 0x7f5f300ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 -- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30158b40 msgr2=0x7f5f30156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30158b40 0x7f5f30156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 -- 172.21.15.7:0/3584328712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0094f0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f35ad4700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d890 tx=0x7f5f20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200125f0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20012770 con 0x7f5f300cb1e0 2023-10-06T11:22:57.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f30157310 con 0x7f5f300cb1e0 2023-10-06T11:22:57.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20012b30 con 0x7f5f300cb1e0 2023-10-06T11:22:57.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.767+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f30146c50 con 0x7f5f300cb1e0 2023-10-06T11:22:57.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.771+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3006ca60 con 0x7f5f300cb1e0 2023-10-06T11:22:57.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.771+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5f2001e460 con 0x7f5f300cb1e0 2023-10-06T11:22:57.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.771+0000 7f5f25ffb700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 0x7f5f14062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:57.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.771+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f5f2007ccd0 con 0x7f5f300cb1e0 2023-10-06T11:22:57.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.771+0000 7f5f352d3700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 0x7f5f14062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:57.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.775+0000 7f5f352d3700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 0x7f5f14062a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f300cd7f0 tx=0x7f5f2c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:57.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.775+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5f20050e00 con 0x7f5f300cb1e0 2023-10-06T11:22:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:57.991+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f5f3006dcc0 con 0x7f5f300cb1e0 2023-10-06T11:22:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f25ffb700 1 -- 172.21.15.7:0/3584328712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5f20050990 con 0x7f5f300cb1e0 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 msgr2=0x7f5f14062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 0x7f5f14062a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f300cd7f0 tx=0x7f5f2c017000 comp rx=0 tx=0).stop 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 msgr2=0x7f5f30146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d890 tx=0x7f5f20008040 comp rx=0 tx=0).stop 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 shutdown_connections 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5f14060580 0x7f5f14062a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f5f30158f10 0x7f5f300ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30158b40 0x7f5f30156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 --2- 172.21.15.7:0/3584328712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300cb1e0 0x7f5f30146710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 >> 172.21.15.7:0/3584328712 conn(0x7f5f30067e30 msgr2=0x7f5f3015b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 shutdown_connections 2023-10-06T11:22:58.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.003+0000 7f5f36ad6700 1 -- 172.21.15.7:0/3584328712 wait complete. 2023-10-06T11:22:58.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T11:22:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 Processor -- start 2023-10-06T11:22:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 -- start start 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4b2c14fdf0 0x7f4b2c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b2c0cc190 0x7f4b2c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c0ae860 con 0x7f4b2c0b00e0 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c157010 con 0x7f4b2c0cc190 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2bfff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c0aeb40 con 0x7f4b2c14fdf0 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b2c0cc190 0x7f4b2c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b2c0b00e0 0x7f4b2c0b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44576/0 (socket says 172.21.15.7:44576) 2023-10-06T11:22:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2b7fe700 1 -- 172.21.15.7:0/4274392553 learned_addr learned my addr 172.21.15.7:0/4274392553 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.127+0000 7f4b2affd700 1 --2- 172.21.15.7:0/4274392553 >> v2:172.21.15.50:3300/0 conn(0x7f4b2c14fdf0 0x7f4b2c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b29ffb700 1 -- 172.21.15.7:0/4274392553 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722517789 0 0) 0x7f4b2c0ae860 con 0x7f4b2c0b00e0 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b29ffb700 1 -- 172.21.15.7:0/4274392553 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b18003570 con 0x7f4b2c0b00e0 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/4274392553 >> v2:172.21.15.50:3300/0 conn(0x7f4b2c14fdf0 msgr2=0x7f4b2c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2a7fc700 1 --2- 172.21.15.7:0/4274392553 >> v2:172.21.15.50:3300/0 conn(0x7f4b2c14fdf0 0x7f4b2c1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/4274392553 >> v1:172.21.15.7:6789/0 conn(0x7f4b2c0b00e0 legacy=0x7f4b2c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/4274392553 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0aee20 con 0x7f4b2c0cc190 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2a7fc700 1 --2- 172.21.15.7:0/4274392553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c156ad0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c06cc00 tx=0x7f4b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f300aa7a157180a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b29ffb700 1 -- 172.21.15.7:0/4274392553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011630 con 0x7f4b2c0cc190 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b29ffb700 1 -- 172.21.15.7:0/4274392553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140117b0 con 0x7f4b2c0cc190 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b29ffb700 1 -- 172.21.15.7:0/4274392553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011b70 con 0x7f4b2c0cc190 2023-10-06T11:22:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 -- 172.21.15.7:0/4274392553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 msgr2=0x7f4b2c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/4274392553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c156ad0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c06cc00 tx=0x7f4b140091a0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 -- 172.21.15.7:0/4274392553 shutdown_connections 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/4274392553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c156ad0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/4274392553 >> v2:172.21.15.50:3300/0 conn(0x7f4b2c14fdf0 0x7f4b2c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 -- 172.21.15.7:0/4274392553 >> 172.21.15.7:0/4274392553 conn(0x7f4b2c0688a0 msgr2=0x7f4b2c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 -- 172.21.15.7:0/4274392553 shutdown_connections 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.131+0000 7f4b2bfff700 1 -- 172.21.15.7:0/4274392553 wait complete. 2023-10-06T11:22:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 Processor -- start 2023-10-06T11:22:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- start start 2023-10-06T11:22:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 0x7f4b2c144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 0x7f4b2c13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c140630 con 0x7f4b2c0cc190 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c140910 con 0x7f4b2c0b00e0 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c144210 con 0x7f4b2c14fdf0 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58138/0 (socket says 172.21.15.7:58138) 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/1950727733 learned_addr learned my addr 172.21.15.7:0/1950727733 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2affd700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 0x7f4b2c144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2b7fe700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 0x7f4b2c13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 msgr2=0x7f4b2c13a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 0x7f4b2c13a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 msgr2=0x7f4b2c144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 0x7f4b2c144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 -- 172.21.15.7:0/1950727733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14009420 con 0x7f4b2c0cc190 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2b7fe700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 0x7f4b2c13a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2affd700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 0x7f4b2c144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2a7fc700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b14013000 tx=0x7f4b14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1401f070 con 0x7f4b2c0cc190 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14011920 con 0x7f4b2c0cc190 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14023410 con 0x7f4b2c0cc190 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c1444f0 con 0x7f4b2c0cc190 2023-10-06T11:22:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.135+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c143ee0 con 0x7f4b2c0cc190 2023-10-06T11:22:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4b1402c3b0 con 0x7f4b2c0cc190 2023-10-06T11:22:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c06c710 con 0x7f4b2c0cc190 2023-10-06T11:22:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b22ffd700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 0x7f4b0c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f4b1407d950 con 0x7f4b2c0cc190 2023-10-06T11:22:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b2affd700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 0x7f4b0c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.139+0000 7f4b2affd700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 0x7f4b0c062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c06b1e0 tx=0x7f4b1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.143+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b1404e030 con 0x7f4b2c0cc190 2023-10-06T11:22:58.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.359+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f4b2c0af100 con 0x7f4b2c0cc190 2023-10-06T11:22:58.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.363+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1950727733 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4b140150a0 con 0x7f4b2c0cc190 2023-10-06T11:22:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 msgr2=0x7f4b0c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 0x7f4b0c062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c06b1e0 tx=0x7f4b1c008040 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 msgr2=0x7f4b2c142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b14013000 tx=0x7f4b14017000 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 shutdown_connections 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4b0c060550 0x7f4b0c062a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4b2c14fdf0 0x7f4b2c13a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c0cc190 0x7f4b2c142d50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 --2- 172.21.15.7:0/1950727733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b2c0b00e0 0x7f4b2c144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 >> 172.21.15.7:0/1950727733 conn(0x7f4b2c0688a0 msgr2=0x7f4b2c148f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 shutdown_connections 2023-10-06T11:22:58.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.367+0000 7f4b2bfff700 1 -- 172.21.15.7:0/1950727733 wait complete. 2023-10-06T11:22:58.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T11:22:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 Processor -- start 2023-10-06T11:22:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 -- start start 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7738153470 0x7f77380caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7738153840 0x7f773815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f773815d0e0 0x7f7738069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773815db90 con 0x7f7738153470 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773815d5d0 con 0x7f7738153840 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773dd90700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773815d8b0 con 0x7f773815d0e0 2023-10-06T11:22:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773cd8e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7738153470 0x7f77380caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773cd8e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7738153470 0x7f77380caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58148/0 (socket says 172.21.15.7:58148) 2023-10-06T11:22:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773cd8e700 1 -- 172.21.15.7:0/1444827999 learned_addr learned my addr 172.21.15.7:0/1444827999 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f7737fff700 1 --2- 172.21.15.7:0/1444827999 >> v2:172.21.15.50:3300/0 conn(0x7f7738153840 0x7f773815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.487+0000 7f773d58f700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f773815d0e0 0x7f7738069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 -- 172.21.15.7:0/1444827999 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f773815d0e0 msgr2=0x7f7738069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f773815d0e0 0x7f7738069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 -- 172.21.15.7:0/1444827999 >> v2:172.21.15.50:3300/0 conn(0x7f7738153840 msgr2=0x7f773815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 --2- 172.21.15.7:0/1444827999 >> v2:172.21.15.50:3300/0 conn(0x7f7738153840 0x7f773815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 -- 172.21.15.7:0/1444827999 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380ca2d0 con 0x7f7738153470 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773cd8e700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7738153470 0x7f77380caad0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f773806db00 tx=0x7f7728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8f9f117fedc4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f77377fe700 1 -- 172.21.15.7:0/1444827999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800b640 con 0x7f7738153470 2023-10-06T11:22:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f77377fe700 1 -- 172.21.15.7:0/1444827999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772800b7a0 con 0x7f7738153470 2023-10-06T11:22:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f77377fe700 1 -- 172.21.15.7:0/1444827999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800bb60 con 0x7f7738153470 2023-10-06T11:22:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 -- 172.21.15.7:0/1444827999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7738153470 msgr2=0x7f77380caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7738153470 0x7f77380caad0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f773806db00 tx=0x7f7728009250 comp rx=0 tx=0).stop 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 -- 172.21.15.7:0/1444827999 shutdown_connections 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f773815d0e0 0x7f7738069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 --2- 172.21.15.7:0/1444827999 >> v2:172.21.15.50:3300/0 conn(0x7f7738153840 0x7f773815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 --2- 172.21.15.7:0/1444827999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7738153470 0x7f77380caad0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 -- 172.21.15.7:0/1444827999 >> 172.21.15.7:0/1444827999 conn(0x7f77380cc190 msgr2=0x7f77380cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 -- 172.21.15.7:0/1444827999 shutdown_connections 2023-10-06T11:22:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.491+0000 7f773dd90700 1 -- 172.21.15.7:0/1444827999 wait complete. 2023-10-06T11:22:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 Processor -- start 2023-10-06T11:22:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 -- start start 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7738153470 0x7f77380b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7738153840 0x7f77380c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380c6680 con 0x7f773815d0e0 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380b08f0 con 0x7f7738153840 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773dd90700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380b0e60 con 0x7f7738153470 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58150/0 (socket says 172.21.15.7:58150) 2023-10-06T11:22:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 -- 172.21.15.7:0/2506474655 learned_addr learned my addr 172.21.15.7:0/2506474655 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f7737fff700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7738153840 0x7f77380c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773cd8e700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7738153470 0x7f77380b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 -- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7738153470 msgr2=0x7f77380b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7738153470 0x7f77380b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 -- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7738153840 msgr2=0x7f77380c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7738153840 0x7f77380c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 -- 172.21.15.7:0/2506474655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280094f0 con 0x7f773815d0e0 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.495+0000 7f773d58f700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7730009d70 tx=0x7f773000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000caa0 con 0x7f773815d0e0 2023-10-06T11:22:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000f040 con 0x7f773815d0e0 2023-10-06T11:22:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773814edd0 con 0x7f773815d0e0 2023-10-06T11:22:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000ce50 con 0x7f773815d0e0 2023-10-06T11:22:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77380b0af0 con 0x7f773815d0e0 2023-10-06T11:22:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773806d240 con 0x7f773815d0e0 2023-10-06T11:22:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7730014690 con 0x7f773815d0e0 2023-10-06T11:22:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.499+0000 7f7734ff9700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 0x7f7724062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.503+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f773007d0d0 con 0x7f773815d0e0 2023-10-06T11:22:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.503+0000 7f773cd8e700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 0x7f7724062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.503+0000 7f773cd8e700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 0x7f7724062a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f773806db00 tx=0x7f7728017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.507+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f773004d770 con 0x7f773815d0e0 2023-10-06T11:22:58.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.727+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f77380b0fa0 con 0x7f773815d0e0 2023-10-06T11:22:58.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f7734ff9700 1 -- 172.21.15.7:0/2506474655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7730050d90 con 0x7f773815d0e0 2023-10-06T11:22:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 msgr2=0x7f7724062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 0x7f7724062a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f773806db00 tx=0x7f7728017000 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 msgr2=0x7f773815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7730009d70 tx=0x7f773000b040 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 shutdown_connections 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7724060580 0x7f7724062a30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773815d0e0 0x7f773815a220 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7738153840 0x7f77380c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 --2- 172.21.15.7:0/2506474655 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7738153470 0x7f77380b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 >> 172.21.15.7:0/2506474655 conn(0x7f77380cc190 msgr2=0x7f7738069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 shutdown_connections 2023-10-06T11:22:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.735+0000 7f773dd90700 1 -- 172.21.15.7:0/2506474655 wait complete. 2023-10-06T11:22:58.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T11:22:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.859+0000 7f1f7289d700 1 Processor -- start 2023-10-06T11:22:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.859+0000 7f1f7289d700 1 -- start start 2023-10-06T11:22:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f6c06bad0 0x7f1f6c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c06e6f0 con 0x7f1f6c06bea0 2023-10-06T11:22:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c1561f0 con 0x7f1f6c06bad0 2023-10-06T11:22:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0cbc70 con 0x7f1f6c155850 2023-10-06T11:22:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f6c06bad0 0x7f1f6c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f6c06bad0 0x7f1f6c06eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58154/0 (socket says 172.21.15.7:58154) 2023-10-06T11:22:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 -- 172.21.15.7:0/2919643170 learned_addr learned my addr 172.21.15.7:0/2919643170 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7209c700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 -- 172.21.15.7:0/2919643170 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 msgr2=0x7f1f6c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 -- 172.21.15.7:0/2919643170 >> v1:172.21.15.50:6789/0 conn(0x7f1f6c06bea0 legacy=0x7f1f6c0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 -- 172.21.15.7:0/2919643170 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0cbf50 con 0x7f1f6c06bad0 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7189b700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c06eb80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bc50 tx=0x7f1f6800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97df3ad121dc90f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f70899700 1 -- 172.21.15.7:0/2919643170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800dbf0 con 0x7f1f6c06bad0 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f70899700 1 -- 172.21.15.7:0/2919643170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800dd50 con 0x7f1f6c06bad0 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f70899700 1 -- 172.21.15.7:0/2919643170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f680196b0 con 0x7f1f6c06bad0 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- 172.21.15.7:0/2919643170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bad0 msgr2=0x7f1f6c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c06eb80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bc50 tx=0x7f1f6800b4f0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- 172.21.15.7:0/2919643170 shutdown_connections 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 --2- 172.21.15.7:0/2919643170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c06eb80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.863+0000 7f1f7289d700 1 -- 172.21.15.7:0/2919643170 >> 172.21.15.7:0/2919643170 conn(0x7f1f6c0cc190 msgr2=0x7f1f6c0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- 172.21.15.7:0/2919643170 shutdown_connections 2023-10-06T11:22:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- 172.21.15.7:0/2919643170 wait complete. 2023-10-06T11:22:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 Processor -- start 2023-10-06T11:22:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- start start 2023-10-06T11:22:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c14fa70 con 0x7f1f6c06bea0 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c3390 con 0x7f1f6c06bad0 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7289d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c3670 con 0x7f1f6c155850 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.867+0000 7f1f7109a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58164/0 (socket says 172.21.15.7:58164) 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 -- 172.21.15.7:0/1778757139 learned_addr learned my addr 172.21.15.7:0/1778757139 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7189b700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7209c700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 -- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 msgr2=0x7f1f6c0c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c0c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 -- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f6c06bad0 msgr2=0x7f1f6c14f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c14f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 -- 172.21.15.7:0/1778757139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6800b790 con 0x7f1f6c06bea0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7209c700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7109a700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c590 tx=0x7f1f5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011590 con 0x7f1f6c06bea0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c011710 con 0x7f1f6c06bea0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c0c3950 con 0x7f1f6c06bea0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011ad0 con 0x7f1f6c06bea0 2023-10-06T11:22:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.871+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c0c3f10 con 0x7f1f6c06bea0 2023-10-06T11:22:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c0c4160 con 0x7f1f6c06bea0 2023-10-06T11:22:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1f5c01b3b0 con 0x7f1f6c06bea0 2023-10-06T11:22:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f61ffb700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 0x7f1f50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1f5c07bb20 con 0x7f1f6c06bea0 2023-10-06T11:22:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f7189b700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 0x7f1f50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.875+0000 7f1f7189b700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 0x7f1f50062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bc50 tx=0x7f1f6801b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:58.883+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1f5c04c1c0 con 0x7f1f6c06bea0 2023-10-06T11:22:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.095+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1f6c06cab0 con 0x7f1f6c06bea0 2023-10-06T11:22:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f61ffb700 1 -- 172.21.15.7:0/1778757139 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1f5c04f7e0 con 0x7f1f6c06bea0 2023-10-06T11:22:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 msgr2=0x7f1f50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 0x7f1f50062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bc50 tx=0x7f1f6801b000 comp rx=0 tx=0).stop 2023-10-06T11:22:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 msgr2=0x7f1f6c1576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c590 tx=0x7f1f5c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 shutdown_connections 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1f50060550 0x7f1f50062a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1f6c155850 0x7f1f6c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6c06bea0 0x7f1f6c1576d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 --2- 172.21.15.7:0/1778757139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f6c06bad0 0x7f1f6c14f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.103+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 >> 172.21.15.7:0/1778757139 conn(0x7f1f6c0cc190 msgr2=0x7f1f6c0c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.107+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 shutdown_connections 2023-10-06T11:22:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.107+0000 7f1f7289d700 1 -- 172.21.15.7:0/1778757139 wait complete. 2023-10-06T11:22:59.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:22:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.223+0000 7fe2809e2700 1 Processor -- start 2023-10-06T11:22:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.223+0000 7fe2809e2700 1 -- start start 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe27c0cb150 0x7fe27c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c06dce0 con 0x7fe27c06a210 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0cb720 con 0x7fe27c0ce650 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0cba00 con 0x7fe27c0cb150 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe279d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe27c06a210 0x7fe27c158e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44586/0 (socket says 172.21.15.7:44586) 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe279d9b700 1 -- 172.21.15.7:0/2674841482 learned_addr learned my addr 172.21.15.7:0/2674841482 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27ad9d700 1 --2- 172.21.15.7:0/2674841482 >> v2:172.21.15.7:3300/0 conn(0x7fe27c0cb150 0x7fe27c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963448165 0 0) 0x7fe27c06dce0 con 0x7fe27c06a210 2023-10-06T11:22:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003570 con 0x7fe27c06a210 2023-10-06T11:22:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525168206 0 0) 0x7fe27c0cb720 con 0x7fe27c0ce650 2023-10-06T11:22:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27ad9d700 1 -- 172.21.15.7:0/2674841482 >> v1:172.21.15.50:6789/0 conn(0x7fe27c0ce650 legacy=0x7fe27c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27ad9d700 1 -- 172.21.15.7:0/2674841482 >> v1:172.21.15.7:6789/0 conn(0x7fe27c06a210 legacy=0x7fe27c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27ad9d700 1 -- 172.21.15.7:0/2674841482 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0cbce0 con 0x7fe27c0cb150 2023-10-06T11:22:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27ad9d700 1 --2- 172.21.15.7:0/2674841482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0cb150 0x7fe27c0cea20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe274009e60 tx=0x7fe274009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bffa8924b592f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400e070 con 0x7fe27c0cb150 2023-10-06T11:22:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27400b850 con 0x7fe27c0cb150 2023-10-06T11:22:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe27959a700 1 -- 172.21.15.7:0/2674841482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270001470 con 0x7fe27c0cb150 2023-10-06T11:22:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- 172.21.15.7:0/2674841482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0cb150 msgr2=0x7fe27c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 --2- 172.21.15.7:0/2674841482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0cb150 0x7fe27c0cea20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe274009e60 tx=0x7fe274009780 comp rx=0 tx=0).stop 2023-10-06T11:22:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- 172.21.15.7:0/2674841482 shutdown_connections 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 --2- 172.21.15.7:0/2674841482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0cb150 0x7fe27c0cea20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- 172.21.15.7:0/2674841482 >> 172.21.15.7:0/2674841482 conn(0x7fe27c068900 msgr2=0x7fe27c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- 172.21.15.7:0/2674841482 shutdown_connections 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.227+0000 7fe2809e2700 1 -- 172.21.15.7:0/2674841482 wait complete. 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 Processor -- start 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- start start 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe27c06a210 0x7fe27c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe27c0cb150 0x7fe27c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0c7de0 con 0x7fe27c0ce650 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0c80c0 con 0x7fe27c06a210 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0c83a0 con 0x7fe27c0cb150 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58182/0 (socket says 172.21.15.7:58182) 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 -- 172.21.15.7:0/534241663 learned_addr learned my addr 172.21.15.7:0/534241663 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27a59c700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe27c06a210 0x7fe27c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe279d9b700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe27c0cb150 0x7fe27c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 -- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe27c0cb150 msgr2=0x7fe27c0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe27c0cb150 0x7fe27c0c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 -- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe27c06a210 msgr2=0x7fe27c153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe27c06a210 0x7fe27c153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 -- 172.21.15.7:0/534241663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274009a20 con 0x7fe27c0ce650 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe27ad9d700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2740097c0 tx=0x7fe274017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27401f410 con 0x7fe27c0ce650 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c15ba20 con 0x7fe27c0ce650 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27401f5b0 con 0x7fe27c0ce650 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27401f970 con 0x7fe27c0ce650 2023-10-06T11:22:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.231+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c0c79b0 con 0x7fe27c0ce650 2023-10-06T11:22:59.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe2740113b0 con 0x7fe27c0ce650 2023-10-06T11:22:59.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe26a7fc700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 0x7fe25c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c15bc10 con 0x7fe27c0ce650 2023-10-06T11:22:59.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe27407ea60 con 0x7fe27c0ce650 2023-10-06T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe27a59c700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 0x7fe25c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.235+0000 7fe27a59c700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 0x7fe25c062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003820 tx=0x7fe26c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.243+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe274023030 con 0x7fe27c0ce650 2023-10-06T11:22:59.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.399+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe27c0c7b40 con 0x7fe27c0ce650 2023-10-06T11:22:59.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.399+0000 7fe26a7fc700 1 -- 172.21.15.7:0/534241663 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13759 (secure 0 0 0) 0x7fe27404f1b0 con 0x7fe27c0ce650 2023-10-06T11:22:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:22:59.405 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:22:56.524362+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:22:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 msgr2=0x7fe25c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 0x7fe25c062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003820 tx=0x7fe26c009040 comp rx=0 tx=0).stop 2023-10-06T11:22:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 msgr2=0x7fe27c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2740097c0 tx=0x7fe274017000 comp rx=0 tx=0).stop 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 shutdown_connections 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe25c060550 0x7fe25c062a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0ce650 0x7fe27c159010 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe27c0cb150 0x7fe27c0c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 --2- 172.21.15.7:0/534241663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe27c06a210 0x7fe27c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 >> 172.21.15.7:0/534241663 conn(0x7fe27c068900 msgr2=0x7fe27c06a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 shutdown_connections 2023-10-06T11:22:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.403+0000 7fe2809e2700 1 -- 172.21.15.7:0/534241663 wait complete. 2023-10-06T11:22:59.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T11:22:59.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:22:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 Processor -- start 2023-10-06T11:22:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 -- start start 2023-10-06T11:22:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 0x7fc6b80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6b80ccca0 0x7fc6b80b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b806de70 con 0x7fc6b8158de0 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b806e0d0 con 0x7fc6b80ccca0 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.531+0000 7fc6bda92700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b806e3b0 con 0x7fc6b81591b0 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6b8158de0 0x7fc6b806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44596/0 (socket says 172.21.15.7:44596) 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b77fe700 1 -- 172.21.15.7:0/3345375022 learned_addr learned my addr 172.21.15.7:0/3345375022 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 --2- 172.21.15.7:0/3345375022 >> v2:172.21.15.7:3300/0 conn(0x7fc6b80ccca0 0x7fc6b80b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6aeffd700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 0x7fc6b80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b6ffd700 1 -- 172.21.15.7:0/3345375022 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328278075 0 0) 0x7fc6b806de70 con 0x7fc6b8158de0 2023-10-06T11:22:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 -- 172.21.15.7:0/3345375022 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 msgr2=0x7fc6b80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 0x7fc6b80cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 -- 172.21.15.7:0/3345375022 >> v1:172.21.15.7:6789/0 conn(0x7fc6b8158de0 legacy=0x7fc6b806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 -- 172.21.15.7:0/3345375022 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b806e690 con 0x7fc6b80ccca0 2023-10-06T11:22:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6aeffd700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 0x7fc6b80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:22:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b7fff700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b80b0050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009e60 tx=0x7fc6b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37559902cc3fd04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b6ffd700 1 -- 172.21.15.7:0/3345375022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000e070 con 0x7fc6b80ccca0 2023-10-06T11:22:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b6ffd700 1 -- 172.21.15.7:0/3345375022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b000bc30 con 0x7fc6b80ccca0 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6b6ffd700 1 -- 172.21.15.7:0/3345375022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0012430 con 0x7fc6b80ccca0 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 -- 172.21.15.7:0/3345375022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 msgr2=0x7fc6b80b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b80b0050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009e60 tx=0x7fc6b0009780 comp rx=0 tx=0).stop 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 -- 172.21.15.7:0/3345375022 shutdown_connections 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b80b0050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 --2- 172.21.15.7:0/3345375022 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b81591b0 0x7fc6b80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 -- 172.21.15.7:0/3345375022 >> 172.21.15.7:0/3345375022 conn(0x7fc6b8067e30 msgr2=0x7fc6b8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 -- 172.21.15.7:0/3345375022 shutdown_connections 2023-10-06T11:22:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.535+0000 7fc6bda92700 1 -- 172.21.15.7:0/3345375022 wait complete. 2023-10-06T11:22:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 Processor -- start 2023-10-06T11:22:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 -- start start 2023-10-06T11:22:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b8158de0 0x7fc6b80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b81591b0 0x7fc6b80b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815b670 con 0x7fc6b80ccca0 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b814a810 con 0x7fc6b81591b0 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6bda92700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815b880 con 0x7fc6b8158de0 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58204/0 (socket says 172.21.15.7:58204) 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.539+0000 7fc6b77fe700 1 -- 172.21.15.7:0/107090344 learned_addr learned my addr 172.21.15.7:0/107090344 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b7fff700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b81591b0 0x7fc6b80b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6aeffd700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b8158de0 0x7fc6b80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 -- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b8158de0 msgr2=0x7fc6b80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b8158de0 0x7fc6b80b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 -- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b81591b0 msgr2=0x7fc6b80b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b81591b0 0x7fc6b80b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 -- 172.21.15.7:0/107090344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b0009a20 con 0x7fc6b80ccca0 2023-10-06T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6b77fe700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c810 tx=0x7fc6a8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005c00 con 0x7fc6b80ccca0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8005d60 con 0x7fc6b80ccca0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b815bb60 con 0x7fc6b80ccca0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8007300 con 0x7fc6b80ccca0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b815c120 con 0x7fc6b80ccca0 2023-10-06T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.543+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b80cd750 con 0x7fc6b80ccca0 2023-10-06T11:22:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.547+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc6a80076b0 con 0x7fc6b80ccca0 2023-10-06T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.547+0000 7fc6affff700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 0x7fc698062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.547+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc6a807bcb0 con 0x7fc6b80ccca0 2023-10-06T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.547+0000 7fc6aeffd700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 0x7fc698062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.547+0000 7fc6aeffd700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 0x7fc698062a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006f00 tx=0x7fc6a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.555+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc6b80cd750 con 0x7fc6b80ccca0 2023-10-06T11:22:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.715+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6b80cd750 con 0x7fc6b80ccca0 2023-10-06T11:22:59.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.715+0000 7fc6affff700 1 -- 172.21.15.7:0/107090344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6b80cd750 con 0x7fc6b80ccca0 2023-10-06T11:22:59.719 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 msgr2=0x7fc698062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 0x7fc698062a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006f00 tx=0x7fc6a0008040 comp rx=0 tx=0).stop 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 msgr2=0x7fc6b814a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c810 tx=0x7fc6a8007dd0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 shutdown_connections 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc698060580 0x7fc698062a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b81591b0 0x7fc6b80b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc6b8158de0 0x7fc6b80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 --2- 172.21.15.7:0/107090344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6b80ccca0 0x7fc6b814a2d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 >> 172.21.15.7:0/107090344 conn(0x7fc6b8067e30 msgr2=0x7fc6b806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 shutdown_connections 2023-10-06T11:22:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.719+0000 7fc6bda92700 1 -- 172.21.15.7:0/107090344 wait complete. 2023-10-06T11:22:59.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:22:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.843+0000 7f13eec95700 1 Processor -- start 2023-10-06T11:22:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.843+0000 7f13eec95700 1 -- start start 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13e806e0e0 0x7f13e80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80cd130 con 0x7f13e80cefe0 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e806e770 con 0x7f13e806e0e0 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80ceb50 con 0x7f13e8159eb0 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13e806e0e0 0x7f13e80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13e806e0e0 0x7f13e80ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58216/0 (socket says 172.21.15.7:58216) 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 -- 172.21.15.7:0/3677320930 learned_addr learned my addr 172.21.15.7:0/3677320930 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eca31700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 -- 172.21.15.7:0/3677320930 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 msgr2=0x7f13e80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 -- 172.21.15.7:0/3677320930 >> v1:172.21.15.50:6789/0 conn(0x7f13e80cefe0 legacy=0x7f13e806d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 -- 172.21.15.7:0/3677320930 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e806e520 con 0x7f13e806e0e0 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13ed232700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80ccbf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cdce0 tx=0x7f13e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84675bee29edb027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13df7fe700 1 -- 172.21.15.7:0/3677320930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000e070 con 0x7f13e806e0e0 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13df7fe700 1 -- 172.21.15.7:0/3677320930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e000ba10 con 0x7f13e806e0e0 2023-10-06T11:22:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13df7fe700 1 -- 172.21.15.7:0/3677320930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00092e0 con 0x7f13e806e0e0 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- 172.21.15.7:0/3677320930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 msgr2=0x7f13e80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80ccbf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cdce0 tx=0x7f13e0009840 comp rx=0 tx=0).stop 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- 172.21.15.7:0/3677320930 shutdown_connections 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80ccbf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 --2- 172.21.15.7:0/3677320930 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- 172.21.15.7:0/3677320930 >> 172.21.15.7:0/3677320930 conn(0x7f13e8067e30 msgr2=0x7f13e8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:22:59.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- 172.21.15.7:0/3677320930 shutdown_connections 2023-10-06T11:22:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.847+0000 7f13eec95700 1 -- 172.21.15.7:0/3677320930 wait complete. 2023-10-06T11:22:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 Processor -- start 2023-10-06T11:22:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 -- start start 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 0x7f13e80ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c9fe0 con 0x7f13e806e0e0 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c7ac0 con 0x7f13e80cefe0 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eec95700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80ae730 con 0x7f13e8159eb0 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13eca31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13ed232700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.851+0000 7f13dffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 0x7f13e80ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13ed232700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51192/0 (socket says 172.21.15.7:51192) 2023-10-06T11:22:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13ed232700 1 -- 172.21.15.7:0/80539398 learned_addr learned my addr 172.21.15.7:0/80539398 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:22:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 -- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 msgr2=0x7f13e80699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 -- 172.21.15.7:0/80539398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 msgr2=0x7f13e80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 -- 172.21.15.7:0/80539398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0009ae0 con 0x7f13e80cefe0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13ed232700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13eca31700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dffff700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 0x7f13e80ae1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cf3d0 tx=0x7f13d8007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800ce60 con 0x7f13e80cefe0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80c93d0 con 0x7f13e80cefe0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8011590 con 0x7f13e80cefe0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8011930 con 0x7f13e80cefe0 2023-10-06T11:22:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80c9a10 con 0x7f13e80cefe0 2023-10-06T11:22:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.855+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80c9d90 con 0x7f13e80cefe0 2023-10-06T11:22:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.859+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f13d8011d20 con 0x7f13e80cefe0 2023-10-06T11:22:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.859+0000 7f13dcff9700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 0x7f13d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:22:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.859+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f13d807ce90 con 0x7f13e80cefe0 2023-10-06T11:22:59.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.859+0000 7f13eca31700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 0x7f13d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:22:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.859+0000 7f13eca31700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 0x7f13d4062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:22:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:22:59.867+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f13d804d490 con 0x7f13e80cefe0 2023-10-06T11:23:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.055+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13e804be00 con 0x7f13e80cefe0 2023-10-06T11:23:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.055+0000 7f13dcff9700 1 -- 172.21.15.7:0/80539398 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13d8050ab0 con 0x7f13e80cefe0 2023-10-06T11:23:00.060 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:23:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 msgr2=0x7f13d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 0x7f13d4062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4008040 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 msgr2=0x7f13e80ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 0x7f13e80ae1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cf3d0 tx=0x7f13d8007ea0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 shutdown_connections 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f13d4060550 0x7f13d4062a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f13e8159eb0 0x7f13e80699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e80cefe0 0x7f13e80ae1f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 --2- 172.21.15.7:0/80539398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e806e0e0 0x7f13e80c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 >> 172.21.15.7:0/80539398 conn(0x7f13e8067e30 msgr2=0x7f13e815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 shutdown_connections 2023-10-06T11:23:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.059+0000 7f13eec95700 1 -- 172.21.15.7:0/80539398 wait complete. 2023-10-06T11:23:00.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:23:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 Processor -- start 2023-10-06T11:23:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 -- start start 2023-10-06T11:23:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c0a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f809c0af6d0 0x7f809c0a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c0a94c0 con 0x7f809c0d0b20 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c0a88b0 con 0x7f809c0af6d0 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a3c24700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0a8b90 con 0x7f809c0b5560 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a21c1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f809c0d0b20 0x7f809c0b3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44608/0 (socket says 172.21.15.7:44608) 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a21c1700 1 -- 172.21.15.7:0/12228628 learned_addr learned my addr 172.21.15.7:0/12228628 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a11bf700 1 --2- 172.21.15.7:0/12228628 >> v2:172.21.15.50:3300/0 conn(0x7f809c0af6d0 0x7f809c0a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a19c0700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c0a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438810426 0 0) 0x7f809c0a94c0 con 0x7f809c0d0b20 2023-10-06T11:23:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8088003570 con 0x7f809c0d0b20 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112177950 0 0) 0x7f8088003570 con 0x7f809c0d0b20 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a11bf700 1 -- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 msgr2=0x7f809c0a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a11bf700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c0a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a11bf700 1 -- 172.21.15.7:0/12228628 >> v1:172.21.15.7:6789/0 conn(0x7f809c0d0b20 legacy=0x7f809c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a11bf700 1 -- 172.21.15.7:0/12228628 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c0a8e70 con 0x7f809c0af6d0 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.167+0000 7f80a19c0700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c0a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a11bf700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c0a9e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80980098e0 tx=0x7f80980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8a3273d7c546a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940017f0 con 0x7f809c0af6d0 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098011420 con 0x7f809c0af6d0 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a09be700 1 -- 172.21.15.7:0/12228628 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980117c0 con 0x7f809c0af6d0 2023-10-06T11:23:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 msgr2=0x7f809c0a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c0a9e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80980098e0 tx=0x7f80980091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- 172.21.15.7:0/12228628 shutdown_connections 2023-10-06T11:23:00.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c0a9e90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- 172.21.15.7:0/12228628 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c0a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- 172.21.15.7:0/12228628 >> 172.21.15.7:0/12228628 conn(0x7f809c0688c0 msgr2=0x7f809c0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- 172.21.15.7:0/12228628 shutdown_connections 2023-10-06T11:23:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- 172.21.15.7:0/12228628 wait complete. 2023-10-06T11:23:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 Processor -- start 2023-10-06T11:23:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- start start 2023-10-06T11:23:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f809c0d0b20 0x7f809c153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15ec10 con 0x7f809c0d0b20 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15afe0 con 0x7f809c0af6d0 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.171+0000 7f80a3c24700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15b2c0 con 0x7f809c0b5560 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a21c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f809c0d0b20 0x7f809c153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a11bf700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:36168/0 (socket says 172.21.15.7:36168) 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 -- 172.21.15.7:0/4007872948 learned_addr learned my addr 172.21.15.7:0/4007872948 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 -- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 msgr2=0x7f809c15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c15e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 -- 172.21.15.7:0/4007872948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f809c0d0b20 msgr2=0x7f809c153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f809c0d0b20 0x7f809c153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 -- 172.21.15.7:0/4007872948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0093c0 con 0x7f809c0af6d0 2023-10-06T11:23:00.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a11bf700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a19c0700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c500 tx=0x7f808c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c015be0 con 0x7f809c0af6d0 2023-10-06T11:23:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8098009420 con 0x7f809c0af6d0 2023-10-06T11:23:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c015d40 con 0x7f809c0af6d0 2023-10-06T11:23:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c005c10 con 0x7f809c0af6d0 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c15b880 con 0x7f809c0af6d0 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c04be00 con 0x7f809c0af6d0 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.175+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f808c006030 con 0x7f809c0af6d0 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.179+0000 7f8091ffb700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 0x7f8080062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.179+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f808c0849f0 con 0x7f809c0af6d0 2023-10-06T11:23:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.179+0000 7f80a11bf700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 0x7f8080062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.179+0000 7f80a11bf700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 0x7f8080062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f809c0afb00 tx=0x7f8098016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.187+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f808c055090 con 0x7f809c0af6d0 2023-10-06T11:23:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.351+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f809c15d9f0 con 0x7f809c0af6d0 2023-10-06T11:23:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.351+0000 7f8091ffb700 1 -- 172.21.15.7:0/4007872948 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+126 (secure 0 0 0) 0x7f808c0586b0 con 0x7f809c0af6d0 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": "rbd" 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 2, 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-10-06T11:23:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:23:00.358 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 msgr2=0x7f8080062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 0x7f8080062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f809c0afb00 tx=0x7f8098016000 comp rx=0 tx=0).stop 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 msgr2=0x7f809c15a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c500 tx=0x7f808c009120 comp rx=0 tx=0).stop 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 shutdown_connections 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8080060550 0x7f8080062a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f809c0d0b20 0x7f809c153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f809c0b5560 0x7f809c15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 --2- 172.21.15.7:0/4007872948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809c0af6d0 0x7f809c15a920 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 >> 172.21.15.7:0/4007872948 conn(0x7f809c0688c0 msgr2=0x7f809c0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 shutdown_connections 2023-10-06T11:23:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.355+0000 7f80a3c24700 1 -- 172.21.15.7:0/4007872948 wait complete. 2023-10-06T11:23:00.370 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:23:00.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:23:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.475+0000 7f7564a9a700 1 Processor -- start 2023-10-06T11:23:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.475+0000 7f7564a9a700 1 -- start start 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f756006c2b0 0x7f756006f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7560156c00 0x7f75600af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756006c820 con 0x7f7560156830 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756006e340 con 0x7f756006c2b0 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756006e620 con 0x7f7560156c00 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7560156830 0x7f75600af0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44610/0 (socket says 172.21.15.7:44610) 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755e59c700 1 -- 172.21.15.7:0/3840245492 learned_addr learned my addr 172.21.15.7:0/3840245492 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 --2- 172.21.15.7:0/3840245492 >> v2:172.21.15.7:3300/0 conn(0x7f756006c2b0 0x7f756006f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755ed9d700 1 --2- 172.21.15.7:0/3840245492 >> v2:172.21.15.50:3300/0 conn(0x7f7560156c00 0x7f75600af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755d59a700 1 -- 172.21.15.7:0/3840245492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601835362 0 0) 0x7f756006c820 con 0x7f7560156830 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755d59a700 1 -- 172.21.15.7:0/3840245492 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c003570 con 0x7f7560156830 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 -- 172.21.15.7:0/3840245492 >> v2:172.21.15.50:3300/0 conn(0x7f7560156c00 msgr2=0x7f75600af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 --2- 172.21.15.7:0/3840245492 >> v2:172.21.15.50:3300/0 conn(0x7f7560156c00 0x7f75600af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 -- 172.21.15.7:0/3840245492 >> v1:172.21.15.7:6789/0 conn(0x7f7560156830 legacy=0x7f75600af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 -- 172.21.15.7:0/3840245492 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756006e900 con 0x7f756006c2b0 2023-10-06T11:23:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755dd9b700 1 --2- 172.21.15.7:0/3840245492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756006c2b0 0x7f756006f100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7560063a60 tx=0x7f75480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37c537840aa33c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755d59a700 1 -- 172.21.15.7:0/3840245492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011630 con 0x7f756006c2b0 2023-10-06T11:23:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755d59a700 1 -- 172.21.15.7:0/3840245492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480117b0 con 0x7f756006c2b0 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f755d59a700 1 -- 172.21.15.7:0/3840245492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011b70 con 0x7f756006c2b0 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- 172.21.15.7:0/3840245492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756006c2b0 msgr2=0x7f756006f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 --2- 172.21.15.7:0/3840245492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756006c2b0 0x7f756006f100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7560063a60 tx=0x7f75480091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- 172.21.15.7:0/3840245492 shutdown_connections 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 --2- 172.21.15.7:0/3840245492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756006c2b0 0x7f756006f100 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 --2- 172.21.15.7:0/3840245492 >> v2:172.21.15.50:3300/0 conn(0x7f7560156c00 0x7f75600af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- 172.21.15.7:0/3840245492 >> 172.21.15.7:0/3840245492 conn(0x7f75600ccde0 msgr2=0x7f75600adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- 172.21.15.7:0/3840245492 shutdown_connections 2023-10-06T11:23:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.479+0000 7f7564a9a700 1 -- 172.21.15.7:0/3840245492 wait complete. 2023-10-06T11:23:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 Processor -- start 2023-10-06T11:23:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- start start 2023-10-06T11:23:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 0x7f7560140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7560156830 0x7f75600acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560143f30 con 0x7f7560156c00 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560143320 con 0x7f7560156830 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7560143600 con 0x7f756006c2b0 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58254/0 (socket says 172.21.15.7:58254) 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 -- 172.21.15.7:0/2579125161 learned_addr learned my addr 172.21.15.7:0/2579125161 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755e59c700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 0x7f7560140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755dd9b700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7560156830 0x7f75600acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 -- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 msgr2=0x7f7560140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 0x7f7560140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 -- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7560156830 msgr2=0x7f75600acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7560156830 0x7f75600acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 -- 172.21.15.7:0/2579125161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548009420 con 0x7f7560156c00 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755e59c700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 0x7f7560140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f755ed9d700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f755800e8d0 tx=0x7f75580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800ee70 con 0x7f7560156c00 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800c850 con 0x7f7560156c00 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75601438e0 con 0x7f7560156c00 2023-10-06T11:23:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800cc50 con 0x7f7560156c00 2023-10-06T11:23:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.483+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75601443c0 con 0x7f7560156c00 2023-10-06T11:23:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.487+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75601440a0 con 0x7f7560156c00 2023-10-06T11:23:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.487+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7558013680 con 0x7f7560156c00 2023-10-06T11:23:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.487+0000 7f75567fc700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 0x7f7540062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.487+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f755807c930 con 0x7f7560156c00 2023-10-06T11:23:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.487+0000 7f755e59c700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 0x7f7540062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.491+0000 7f755e59c700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 0x7f7540062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7550005c50 tx=0x7f7550009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.495+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f755804cf30 con 0x7f7560156c00 2023-10-06T11:23:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:00.659+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f756004be00 con 0x7f7560156c00 2023-10-06T11:23:01.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.543+0000 7f75567fc700 1 -- 172.21.15.7:0/2579125161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7558050550 con 0x7f7560156c00 2023-10-06T11:23:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 msgr2=0x7f7540062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 0x7f7540062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7550005c50 tx=0x7f7550009040 comp rx=0 tx=0).stop 2023-10-06T11:23:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 msgr2=0x7f75600ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f755800e8d0 tx=0x7f75580091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 shutdown_connections 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7540060550 0x7f7540062a00 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7550005c50 tx=0x7f7550009040 comp rx=0 tx=0).stop 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7560156c00 0x7f75600ad3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7560156830 0x7f75600acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 --2- 172.21.15.7:0/2579125161 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f756006c2b0 0x7f7560140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.551+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 >> 172.21.15.7:0/2579125161 conn(0x7f75600ccde0 msgr2=0x7f7560153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.555+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 shutdown_connections 2023-10-06T11:23:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.555+0000 7f7564a9a700 1 -- 172.21.15.7:0/2579125161 wait complete. 2023-10-06T11:23:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-10-06T11:23:01.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:23:01.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 Processor -- start 2023-10-06T11:23:01.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 -- start start 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba40cd130 con 0x7f3ba40cefe0 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba406e770 con 0x7f3ba4159eb0 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba8e5b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40ceb50 con 0x7f3ba406e0e0 2023-10-06T11:23:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba259c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51226/0 (socket says 172.21.15.7:51226) 2023-10-06T11:23:01.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.683+0000 7f3ba259c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:36194/0 (socket says 172.21.15.7:36194) 2023-10-06T11:23:01.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba2d9d700 1 -- 172.21.15.7:0/2895254418 learned_addr learned my addr 172.21.15.7:0/2895254418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba259c700 1 -- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 msgr2=0x7f3ba40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba259c700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba259c700 1 -- 172.21.15.7:0/2895254418 >> v1:172.21.15.50:6789/0 conn(0x7f3ba40cefe0 legacy=0x7f3ba406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba159a700 1 -- 172.21.15.7:0/2895254418 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228978013 0 0) 0x7f3ba40cd130 con 0x7f3ba40cefe0 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba259c700 1 -- 172.21.15.7:0/2895254418 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba406e4f0 con 0x7f3ba4159eb0 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba2d9d700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:01.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba259c700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098e0 tx=0x7f3b940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cccf85236536523 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba159a700 1 -- 172.21.15.7:0/2895254418 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400b640 con 0x7f3ba4159eb0 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba159a700 1 -- 172.21.15.7:0/2895254418 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9400b7a0 con 0x7f3ba4159eb0 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba159a700 1 -- 172.21.15.7:0/2895254418 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400bb40 con 0x7f3ba4159eb0 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba4159eb0 msgr2=0x7f3ba40cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098e0 tx=0x7f3b940091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2895254418 shutdown_connections 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2895254418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba4159eb0 0x7f3ba40cc6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2895254418 >> 172.21.15.7:0/2895254418 conn(0x7f3ba4067e30 msgr2=0x7f3ba4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2895254418 shutdown_connections 2023-10-06T11:23:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.687+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2895254418 wait complete. 2023-10-06T11:23:01.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 Processor -- start 2023-10-06T11:23:01.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 -- start start 2023-10-06T11:23:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba4158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba40cefe0 0x7f3ba415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40c7fd0 con 0x7f3ba4159eb0 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41589d0 con 0x7f3ba40cefe0 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba8e5b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40c73c0 con 0x7f3ba406e0e0 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58266/0 (socket says 172.21.15.7:58266) 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 -- 172.21.15.7:0/2196442439 learned_addr learned my addr 172.21.15.7:0/2196442439 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba259c700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba4158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba1d9b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba40cefe0 0x7f3ba415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 -- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 msgr2=0x7f3ba4158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba4158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 -- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba40cefe0 msgr2=0x7f3ba415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba40cefe0 0x7f3ba415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.691+0000 7f3ba2d9d700 1 -- 172.21.15.7:0/2196442439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009ae0 con 0x7f3ba4159eb0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3ba2d9d700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009f30 tx=0x7f3b9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c018070 con 0x7f3ba4159eb0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00f040 con 0x7f3ba4159eb0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b94009420 con 0x7f3ba4159eb0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00cb70 con 0x7f3ba4159eb0 2023-10-06T11:23:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba40c7980 con 0x7f3ba4159eb0 2023-10-06T11:23:01.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.695+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba40c8140 con 0x7f3ba4159eb0 2023-10-06T11:23:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.699+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3b9c003020 con 0x7f3ba4159eb0 2023-10-06T11:23:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.699+0000 7f3b927fc700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 0x7f3b84062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.699+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f3b9c07da40 con 0x7f3ba4159eb0 2023-10-06T11:23:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.699+0000 7f3ba259c700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 0x7f3b84062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.699+0000 7f3ba259c700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 0x7f3b84062a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b940093f0 tx=0x7f3b94017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.707+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3b9c04df70 con 0x7f3ba4159eb0 2023-10-06T11:23:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:01.871+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ba404be00 con 0x7f3ba4159eb0 2023-10-06T11:23:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.559+0000 7f3b927fc700 1 -- 172.21.15.7:0/2196442439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3b9c051590 con 0x7f3ba4159eb0 2023-10-06T11:23:02.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 msgr2=0x7f3b84062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 0x7f3b84062a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b940093f0 tx=0x7f3b94017000 comp rx=0 tx=0).stop 2023-10-06T11:23:02.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 msgr2=0x7f3ba416a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009f30 tx=0x7f3b9c00b040 comp rx=0 tx=0).stop 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 shutdown_connections 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3b84060580 0x7f3b84062a30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba4159eb0 0x7f3ba416a230 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ba40cefe0 0x7f3ba415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 --2- 172.21.15.7:0/2196442439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3ba406e0e0 0x7f3ba4158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 >> 172.21.15.7:0/2196442439 conn(0x7f3ba4067e30 msgr2=0x7f3ba40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 shutdown_connections 2023-10-06T11:23:02.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.583+0000 7f3ba8e5b700 1 -- 172.21.15.7:0/2196442439 wait complete. 2023-10-06T11:23:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-10-06T11:23:02.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:23:02.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.703+0000 7f0e0b1f8700 1 Processor -- start 2023-10-06T11:23:02.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.703+0000 7f0e0b1f8700 1 -- start start 2023-10-06T11:23:02.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0b1f8700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0e04158c60 0x7f0e0406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0b1f8700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 0x7f0e0406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0b1f8700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e040c7980 con 0x7f0e04067e30 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0b1f8700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0406cd70 con 0x7f0e04158c60 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0b1f8700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0406d050 con 0x7f0e04159030 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 0x7f0e0406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0a1f6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0e04158c60 0x7f0e0406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0a9f7700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0e04067e30 0x7f0e0415c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:56112/0 (socket says 172.21.15.7:56112) 2023-10-06T11:23:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e0a9f7700 1 -- 172.21.15.7:0/3816101658 learned_addr learned my addr 172.21.15.7:0/3816101658 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e091f4700 1 -- 172.21.15.7:0/3816101658 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987326493 0 0) 0x7f0e040c7980 con 0x7f0e04067e30 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 -- 172.21.15.7:0/3816101658 >> v2:172.21.15.50:3300/0 conn(0x7f0e04158c60 msgr2=0x7f0e0406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 --2- 172.21.15.7:0/3816101658 >> v2:172.21.15.50:3300/0 conn(0x7f0e04158c60 0x7f0e0406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 -- 172.21.15.7:0/3816101658 >> v1:172.21.15.50:6789/0 conn(0x7f0e04067e30 legacy=0x7f0e0415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 -- 172.21.15.7:0/3816101658 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e040c7610 con 0x7f0e04159030 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e099f5700 1 --2- 172.21.15.7:0/3816101658 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 0x7f0e0406c790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098e0 tx=0x7f0df80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147e45810a1170f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e091f4700 1 -- 172.21.15.7:0/3816101658 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800b640 con 0x7f0e04159030 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e091f4700 1 -- 172.21.15.7:0/3816101658 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df800b7a0 con 0x7f0e04159030 2023-10-06T11:23:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.707+0000 7f0e091f4700 1 -- 172.21.15.7:0/3816101658 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800bb40 con 0x7f0e04159030 2023-10-06T11:23:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/3816101658 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 msgr2=0x7f0e0406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/3816101658 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 0x7f0e0406c790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098e0 tx=0x7f0df80091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/3816101658 shutdown_connections 2023-10-06T11:23:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/3816101658 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04159030 0x7f0e0406c790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/3816101658 >> v2:172.21.15.50:3300/0 conn(0x7f0e04158c60 0x7f0e0406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/3816101658 >> 172.21.15.7:0/3816101658 conn(0x7f0e0406f2a0 msgr2=0x7f0e04068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/3816101658 shutdown_connections 2023-10-06T11:23:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/3816101658 wait complete. 2023-10-06T11:23:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 Processor -- start 2023-10-06T11:23:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.711+0000 7f0e0b1f8700 1 -- start start 2023-10-06T11:23:02.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04158c60 0x7f0e040aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e040ab170 0x7f0e040c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e040c6070 con 0x7f0e04067e30 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e040c51e0 con 0x7f0e040ab170 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e040c5460 con 0x7f0e04158c60 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54062/0 (socket says 172.21.15.7:54062) 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 -- 172.21.15.7:0/2642274507 learned_addr learned my addr 172.21.15.7:0/2642274507 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e099f5700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04158c60 0x7f0e040aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a9f7700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e040ab170 0x7f0e040c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 -- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04158c60 msgr2=0x7f0e040aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04158c60 0x7f0e040aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 -- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e040ab170 msgr2=0x7f0e040c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e040ab170 0x7f0e040c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 -- 172.21.15.7:0/2642274507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df8009420 con 0x7f0e04067e30 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0a1f6700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000ecb0 tx=0x7f0e000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000c850 con 0x7f0e04067e30 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000c9b0 con 0x7f0e04067e30 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e040c5740 con 0x7f0e04067e30 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000cd50 con 0x7f0e04067e30 2023-10-06T11:23:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.715+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e040c5d80 con 0x7f0e04067e30 2023-10-06T11:23:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e040af2f0 con 0x7f0e04067e30 2023-10-06T11:23:02.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0e00013680 con 0x7f0e04067e30 2023-10-06T11:23:02.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0df67fc700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 0x7f0de8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:02.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f0e0007ccd0 con 0x7f0e04067e30 2023-10-06T11:23:02.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0e099f5700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 0x7f0de8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.719+0000 7f0e099f5700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 0x7f0de8062a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0df80091e0 tx=0x7f0df8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:02.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.727+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0e0004cfe0 con 0x7f0e04067e30 2023-10-06T11:23:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:02.927+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0e040c7300 con 0x7f0e04067e30 2023-10-06T11:23:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.587+0000 7f0df67fc700 1 -- 172.21.15.7:0/2642274507 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0e00050600 con 0x7f0e04067e30 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 msgr2=0x7f0de8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 0x7f0de8062a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0df80091e0 tx=0x7f0df8017000 comp rx=0 tx=0).stop 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 msgr2=0x7f0e040aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000ecb0 tx=0x7f0e000091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 shutdown_connections 2023-10-06T11:23:03.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0de8060550 0x7f0de8062a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e040ab170 0x7f0e040c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0e04158c60 0x7f0e040aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 --2- 172.21.15.7:0/2642274507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e04067e30 0x7f0e040aec90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 >> 172.21.15.7:0/2642274507 conn(0x7f0e0406f2a0 msgr2=0x7f0e0406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 shutdown_connections 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.603+0000 7f0e0b1f8700 1 -- 172.21.15.7:0/2642274507 wait complete. 2023-10-06T11:23:03.608 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2023-10-06T11:23:03.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.719+0000 7fc659bff700 1 Processor -- start 2023-10-06T11:23:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.719+0000 7fc659bff700 1 -- start start 2023-10-06T11:23:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.719+0000 7fc659bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc654155e30 0x7fc6540c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.719+0000 7fc659bff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 0x7fc65406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 0x7fc6540c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540cb6b0 con 0x7fc654155e30 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65406ebb0 con 0x7fc654156200 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cc140 con 0x7fc65406e770 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc654155e30 0x7fc6540c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc654155e30 0x7fc6540c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54068/0 (socket says 172.21.15.7:54068) 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 -- 172.21.15.7:0/3373981717 learned_addr learned my addr 172.21.15.7:0/3373981717 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc652ffd700 1 --2- 172.21.15.7:0/3373981717 >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 0x7fc65406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc653fff700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 0x7fc6540c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 -- 172.21.15.7:0/3373981717 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 msgr2=0x7fc6540c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 0x7fc6540c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 -- 172.21.15.7:0/3373981717 >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 msgr2=0x7fc65406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 --2- 172.21.15.7:0/3373981717 >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 0x7fc65406e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 -- 172.21.15.7:0/3373981717 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654156780 con 0x7fc654155e30 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc653fff700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 0x7fc6540c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc652ffd700 1 --2- 172.21.15.7:0/3373981717 >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 0x7fc65406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6537fe700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc654155e30 0x7fc6540c8680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6540694d0 tx=0x7fc644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5d076401ad01c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6527fc700 1 -- 172.21.15.7:0/3373981717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400b860 con 0x7fc654155e30 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6527fc700 1 -- 172.21.15.7:0/3373981717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64400b9c0 con 0x7fc654155e30 2023-10-06T11:23:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc6527fc700 1 -- 172.21.15.7:0/3373981717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400bd60 con 0x7fc654155e30 2023-10-06T11:23:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- 172.21.15.7:0/3373981717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc654155e30 msgr2=0x7fc6540c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc654155e30 0x7fc6540c8680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6540694d0 tx=0x7fc644009250 comp rx=0 tx=0).stop 2023-10-06T11:23:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- 172.21.15.7:0/3373981717 shutdown_connections 2023-10-06T11:23:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc65406e770 0x7fc6540c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 --2- 172.21.15.7:0/3373981717 >> v2:172.21.15.50:3300/0 conn(0x7fc654156200 0x7fc65406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 --2- 172.21.15.7:0/3373981717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc654155e30 0x7fc6540c8680 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- 172.21.15.7:0/3373981717 >> 172.21.15.7:0/3373981717 conn(0x7fc654067e30 msgr2=0x7fc654068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- 172.21.15.7:0/3373981717 shutdown_connections 2023-10-06T11:23:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.723+0000 7fc659bff700 1 -- 172.21.15.7:0/3373981717 wait complete. 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 Processor -- start 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 -- start start 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654155e30 0x7fc654158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc654156200 0x7fc654152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654168e20 con 0x7fc65406e770 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541596a0 con 0x7fc654155e30 2023-10-06T11:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc659bff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654159980 con 0x7fc654156200 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54076/0 (socket says 172.21.15.7:54076) 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 -- 172.21.15.7:0/567475277 learned_addr learned my addr 172.21.15.7:0/567475277 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc653fff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc654156200 0x7fc654152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc652ffd700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654155e30 0x7fc654158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 -- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc654156200 msgr2=0x7fc654152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc654156200 0x7fc654152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 -- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654155e30 msgr2=0x7fc654158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654155e30 0x7fc654158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 -- 172.21.15.7:0/567475277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440094f0 con 0x7fc65406e770 2023-10-06T11:23:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.727+0000 7fc6537fe700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc644009290 tx=0x7fc644017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64401d640 con 0x7fc65406e770 2023-10-06T11:23:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644021070 con 0x7fc65406e770 2023-10-06T11:23:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc654159c60 con 0x7fc65406e770 2023-10-06T11:23:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64401d9f0 con 0x7fc65406e770 2023-10-06T11:23:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65415a220 con 0x7fc65406e770 2023-10-06T11:23:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65404be00 con 0x7fc65406e770 2023-10-06T11:23:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc644028480 con 0x7fc65406e770 2023-10-06T11:23:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 0x7fc640062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc644023070 con 0x7fc65406e770 2023-10-06T11:23:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.731+0000 7fc652ffd700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 0x7fc640062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.735+0000 7fc652ffd700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 0x7fc640062a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc6540c8f00 tx=0x7fc648008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.739+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc644050e20 con 0x7fc65406e770 2023-10-06T11:23:03.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.899+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6540cb8a0 con 0x7fc65406e770 2023-10-06T11:23:03.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.899+0000 7fc63b7fe700 1 -- 172.21.15.7:0/567475277 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16988 (secure 0 0 0) 0x7fc644015090 con 0x7fc65406e770 2023-10-06T11:23:03.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:03.906 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:02.930412+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 msgr2=0x7fc640062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 0x7fc640062a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc6540c8f00 tx=0x7fc648008040 comp rx=0 tx=0).stop 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 msgr2=0x7fc6541688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc644009290 tx=0x7fc644017000 comp rx=0 tx=0).stop 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 shutdown_connections 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc640060580 0x7fc640062a30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc654156200 0x7fc654152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654155e30 0x7fc654158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 --2- 172.21.15.7:0/567475277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406e770 0x7fc6541688e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 >> 172.21.15.7:0/567475277 conn(0x7fc654067e30 msgr2=0x7fc654154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 shutdown_connections 2023-10-06T11:23:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:03.903+0000 7fc659bff700 1 -- 172.21.15.7:0/567475277 wait complete. 2023-10-06T11:23:03.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 Processor -- start 2023-10-06T11:23:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 -- start start 2023-10-06T11:23:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1dec0af700 0x7f1dec0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1dec0b11c0 0x7f1dec15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dec0b07f0 con 0x7f1dec158df0 2023-10-06T11:23:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dec0afbe0 con 0x7f1dec0b11c0 2023-10-06T11:23:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dec0afec0 con 0x7f1dec0af700 2023-10-06T11:23:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1dec0b11c0 0x7f1dec15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1dec0b11c0 0x7f1dec15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54086/0 (socket says 172.21.15.7:54086) 2023-10-06T11:23:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 -- 172.21.15.7:0/3352388765 learned_addr learned my addr 172.21.15.7:0/3352388765 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df0b5a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1dec158df0 0x7f1dec0b3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49712/0 (socket says 172.21.15.7:49712) 2023-10-06T11:23:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1debfff700 1 --2- 172.21.15.7:0/3352388765 >> v2:172.21.15.50:3300/0 conn(0x7f1dec0af700 0x7f1dec0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 -- 172.21.15.7:0/3352388765 >> v2:172.21.15.50:3300/0 conn(0x7f1dec0af700 msgr2=0x7f1dec0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:04.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 --2- 172.21.15.7:0/3352388765 >> v2:172.21.15.50:3300/0 conn(0x7f1dec0af700 0x7f1dec0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 -- 172.21.15.7:0/3352388765 >> v1:172.21.15.7:6789/0 conn(0x7f1dec158df0 legacy=0x7f1dec0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 -- 172.21.15.7:0/3352388765 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0b01a0 con 0x7f1dec0b11c0 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df135b700 1 --2- 172.21.15.7:0/3352388765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec0b11c0 0x7f1dec15caf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1dec069190 tx=0x7f1de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5940330fa6e6abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1deb7fe700 1 -- 172.21.15.7:0/3352388765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4009e70 con 0x7f1dec0b11c0 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1deb7fe700 1 -- 172.21.15.7:0/3352388765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de400b860 con 0x7f1dec0b11c0 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1deb7fe700 1 -- 172.21.15.7:0/3352388765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400bc00 con 0x7f1dec0b11c0 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.023+0000 7f1df2dbe700 1 -- 172.21.15.7:0/3352388765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec0b11c0 msgr2=0x7f1dec15caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/3352388765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec0b11c0 0x7f1dec15caf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1dec069190 tx=0x7f1de4009250 comp rx=0 tx=0).stop 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- 172.21.15.7:0/3352388765 shutdown_connections 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/3352388765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec0b11c0 0x7f1dec15caf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/3352388765 >> v2:172.21.15.50:3300/0 conn(0x7f1dec0af700 0x7f1dec0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- 172.21.15.7:0/3352388765 >> 172.21.15.7:0/3352388765 conn(0x7f1dec0688c0 msgr2=0x7f1dec0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- 172.21.15.7:0/3352388765 shutdown_connections 2023-10-06T11:23:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- 172.21.15.7:0/3352388765 wait complete. 2023-10-06T11:23:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 Processor -- start 2023-10-06T11:23:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- start start 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dec0af700 0x7f1dec0a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 0x7f1dec1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0ab2a0 con 0x7f1dec158df0 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec15a5a0 con 0x7f1dec0af700 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df2dbe700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec159c70 con 0x7f1dec0b11c0 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df135b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1debfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 0x7f1dec1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df0b5a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dec0af700 0x7f1dec0a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df135b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54092/0 (socket says 172.21.15.7:54092) 2023-10-06T11:23:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.027+0000 7f1df135b700 1 -- 172.21.15.7:0/1313395147 learned_addr learned my addr 172.21.15.7:0/1313395147 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 -- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dec0af700 msgr2=0x7f1dec0a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dec0af700 0x7f1dec0a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 -- 172.21.15.7:0/1313395147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 msgr2=0x7f1dec0abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 -- 172.21.15.7:0/1313395147 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de40094f0 con 0x7f1dec0b11c0 2023-10-06T11:23:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1df135b700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1debfff700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 0x7f1dec1478b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009ec0 tx=0x7f1dd400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400cc00 con 0x7f1dec0b11c0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400f040 con 0x7f1dec0b11c0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec159f50 con 0x7f1dec0b11c0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4014480 con 0x7f1dec0b11c0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec147df0 con 0x7f1dec0b11c0 2023-10-06T11:23:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.031+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec15a710 con 0x7f1dec0b11c0 2023-10-06T11:23:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.035+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1dd4014830 con 0x7f1dec0b11c0 2023-10-06T11:23:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.035+0000 7f1de8ff9700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 0x7f1dd0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.035+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1dd407d770 con 0x7f1dec0b11c0 2023-10-06T11:23:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.035+0000 7f1df0b5a700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 0x7f1dd0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.035+0000 7f1df0b5a700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 0x7f1dd0062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc004fd0 tx=0x7f1ddc007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.047+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1dd404da20 con 0x7f1dec0b11c0 2023-10-06T11:23:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.211+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dec04be00 con 0x7f1dec0b11c0 2023-10-06T11:23:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.215+0000 7f1de8ff9700 1 -- 172.21.15.7:0/1313395147 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f1dd4051040 con 0x7f1dec0b11c0 2023-10-06T11:23:04.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:04.220 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:02.930412+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 msgr2=0x7f1dd0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 0x7f1dd0062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc004fd0 tx=0x7f1ddc007eb0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 msgr2=0x7f1dec1478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 0x7f1dec1478b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009ec0 tx=0x7f1dd400b040 comp rx=0 tx=0).stop 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 shutdown_connections 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1dd0060550 0x7f1dd0062a00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dec158df0 0x7f1dec0abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1dec0b11c0 0x7f1dec1478b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 --2- 172.21.15.7:0/1313395147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dec0af700 0x7f1dec0a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 >> 172.21.15.7:0/1313395147 conn(0x7f1dec0688c0 msgr2=0x7f1dec06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 shutdown_connections 2023-10-06T11:23:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.219+0000 7f1df2dbe700 1 -- 172.21.15.7:0/1313395147 wait complete. 2023-10-06T11:23:04.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:23:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.339+0000 7f809cb42700 1 Processor -- start 2023-10-06T11:23:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.339+0000 7f809cb42700 1 -- start start 2023-10-06T11:23:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f80981550f0 0x7f80980ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80981554c0 0x7f8098069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980c7af0 con 0x7f80980cae10 2023-10-06T11:23:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80980cb1e0 con 0x7f80981554c0 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80980cb3e0 con 0x7f80981550f0 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8097fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f80980cae10 0x7f8098158e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49716/0 (socket says 172.21.15.7:49716) 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8097fff700 1 -- 172.21.15.7:0/600067896 learned_addr learned my addr 172.21.15.7:0/600067896 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 --2- 172.21.15.7:0/600067896 >> v2:172.21.15.7:3300/0 conn(0x7f80981554c0 0x7f8098069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f80977fe700 1 --2- 172.21.15.7:0/600067896 >> v2:172.21.15.50:3300/0 conn(0x7f80981550f0 0x7f80980ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8096ffd700 1 -- 172.21.15.7:0/600067896 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955722302 0 0) 0x7f80980c7af0 con 0x7f80980cae10 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8096ffd700 1 -- 172.21.15.7:0/600067896 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8084004680 con 0x7f80980cae10 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 -- 172.21.15.7:0/600067896 >> v2:172.21.15.50:3300/0 conn(0x7f80981550f0 msgr2=0x7f80980ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 --2- 172.21.15.7:0/600067896 >> v2:172.21.15.50:3300/0 conn(0x7f80981550f0 0x7f80980ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 -- 172.21.15.7:0/600067896 >> v1:172.21.15.7:6789/0 conn(0x7f80980cae10 legacy=0x7f8098158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 -- 172.21.15.7:0/600067896 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980cb6c0 con 0x7f80981554c0 2023-10-06T11:23:04.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f808ffff700 1 --2- 172.21.15.7:0/600067896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981554c0 0x7f8098069cd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f809806dcf0 tx=0x7f80800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604dd39aa9c8e530 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8096ffd700 1 -- 172.21.15.7:0/600067896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080011630 con 0x7f80981554c0 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8096ffd700 1 -- 172.21.15.7:0/600067896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80800117b0 con 0x7f80981554c0 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f8096ffd700 1 -- 172.21.15.7:0/600067896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080011b70 con 0x7f80981554c0 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- 172.21.15.7:0/600067896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981554c0 msgr2=0x7f8098069cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 --2- 172.21.15.7:0/600067896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981554c0 0x7f8098069cd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f809806dcf0 tx=0x7f80800091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- 172.21.15.7:0/600067896 shutdown_connections 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 --2- 172.21.15.7:0/600067896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981554c0 0x7f8098069cd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 --2- 172.21.15.7:0/600067896 >> v2:172.21.15.50:3300/0 conn(0x7f80981550f0 0x7f80980ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.343+0000 7f809cb42700 1 -- 172.21.15.7:0/600067896 >> 172.21.15.7:0/600067896 conn(0x7f8098067e30 msgr2=0x7f8098068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- 172.21.15.7:0/600067896 shutdown_connections 2023-10-06T11:23:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- 172.21.15.7:0/600067896 wait complete. 2023-10-06T11:23:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 Processor -- start 2023-10-06T11:23:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- start start 2023-10-06T11:23:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80980cae10 0x7f80981536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f80980b12e0 0x7f80980b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098153260 con 0x7f80981550f0 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80981586d0 con 0x7f80980cae10 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f809cb42700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80980b1e50 con 0x7f80980b12e0 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.347+0000 7f808ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54114/0 (socket says 172.21.15.7:54114) 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 -- 172.21.15.7:0/2829312250 learned_addr learned my addr 172.21.15.7:0/2829312250 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f8097fff700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f80980b12e0 0x7f80980b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f80977fe700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80980cae10 0x7f80981536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 -- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f80980b12e0 msgr2=0x7f80980b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f80980b12e0 0x7f80980b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 -- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80980cae10 msgr2=0x7f80981536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80980cae10 0x7f80981536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 -- 172.21.15.7:0/2829312250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009420 con 0x7f80981550f0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808ffff700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8080013000 tx=0x7f8080017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808001f070 con 0x7f80981550f0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080000bc0 con 0x7f80981550f0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8098152930 con 0x7f80981550f0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080011b50 con 0x7f80981550f0 2023-10-06T11:23:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8098152ef0 con 0x7f80981550f0 2023-10-06T11:23:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8098153410 con 0x7f80981550f0 2023-10-06T11:23:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8080023070 con 0x7f80981550f0 2023-10-06T11:23:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.351+0000 7f808f7fe700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 0x7f8078062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.355+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f808007dc30 con 0x7f80981550f0 2023-10-06T11:23:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.355+0000 7f80977fe700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 0x7f8078062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:04.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.355+0000 7f80977fe700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 0x7f8078062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8088009db0 tx=0x7f80880091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:04.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.359+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8098153410 con 0x7f80981550f0 2023-10-06T11:23:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:04.555+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80981533a0 con 0x7f80981550f0 2023-10-06T11:23:05.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:05.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:05.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.235+0000 7f808f7fe700 1 -- 172.21.15.7:0/2829312250 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f80800150b0 con 0x7f80981550f0 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 msgr2=0x7f8078062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 0x7f8078062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8088009db0 tx=0x7f80880091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 msgr2=0x7f809814ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8080013000 tx=0x7f8080017000 comp rx=0 tx=0).stop 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 shutdown_connections 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8078060550 0x7f8078062a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f80980b12e0 0x7f80980b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80981550f0 0x7f809814ed60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 --2- 172.21.15.7:0/2829312250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80980cae10 0x7f80981536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 >> 172.21.15.7:0/2829312250 conn(0x7f8098067e30 msgr2=0x7f80980c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 shutdown_connections 2023-10-06T11:23:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.239+0000 7f809cb42700 1 -- 172.21.15.7:0/2829312250 wait complete. 2023-10-06T11:23:05.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:23:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 Processor -- start 2023-10-06T11:23:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 -- start start 2023-10-06T11:23:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b41e0 0x7f85fc0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85fc06de10 0x7f85fc159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0b1190 con 0x7f85fc0b3e10 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0b0f80 con 0x7f85fc06de10 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0b1470 con 0x7f85fc0b41e0 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85fc06de10 0x7f85fc159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85fc06de10 0x7f85fc159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54128/0 (socket says 172.21.15.7:54128) 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.375+0000 7f85fad9d700 1 -- 172.21.15.7:0/2530752638 learned_addr learned my addr 172.21.15.7:0/2530752638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85f9d9b700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b41e0 0x7f85fc0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fad9d700 1 -- 172.21.15.7:0/2530752638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b41e0 msgr2=0x7f85fc0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fad9d700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b41e0 0x7f85fc0b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fad9d700 1 -- 172.21.15.7:0/2530752638 >> v1:172.21.15.50:6789/0 conn(0x7f85fc0b3e10 legacy=0x7f85fc06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fad9d700 1 -- 172.21.15.7:0/2530752638 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0b1750 con 0x7f85fc06de10 2023-10-06T11:23:05.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fad9d700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc159870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009f20 tx=0x7f85f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda0e452c55b9ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85f959a700 1 -- 172.21.15.7:0/2530752638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400e070 con 0x7f85fc06de10 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85f959a700 1 -- 172.21.15.7:0/2530752638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400ba80 con 0x7f85fc06de10 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85f959a700 1 -- 172.21.15.7:0/2530752638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40092e0 con 0x7f85fc06de10 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 -- 172.21.15.7:0/2530752638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 msgr2=0x7f85fc159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc159870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009f20 tx=0x7f85f4009840 comp rx=0 tx=0).stop 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 -- 172.21.15.7:0/2530752638 shutdown_connections 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc159870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 --2- 172.21.15.7:0/2530752638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b41e0 0x7f85fc0b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 -- 172.21.15.7:0/2530752638 >> 172.21.15.7:0/2530752638 conn(0x7f85fc0cfd00 msgr2=0x7f85fc0d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 -- 172.21.15.7:0/2530752638 shutdown_connections 2023-10-06T11:23:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.379+0000 7f85fb59e700 1 -- 172.21.15.7:0/2530752638 wait complete. 2023-10-06T11:23:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 Processor -- start 2023-10-06T11:23:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- start start 2023-10-06T11:23:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b3e10 0x7f85fc16d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85fc0b41e0 0x7f85fc169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0a88b0 con 0x7f85fc06de10 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc16a960 con 0x7f85fc0b41e0 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0c64f0 con 0x7f85fc0b3e10 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54132/0 (socket says 172.21.15.7:54132) 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 -- 172.21.15.7:0/1880910220 learned_addr learned my addr 172.21.15.7:0/1880910220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85f9d9b700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b3e10 0x7f85fc16d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fad9d700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85fc0b41e0 0x7f85fc169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 -- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b3e10 msgr2=0x7f85fc16d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b3e10 0x7f85fc16d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 -- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85fc0b41e0 msgr2=0x7f85fc169810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85fc0b41e0 0x7f85fc169810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 -- 172.21.15.7:0/1880910220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009ae0 con 0x7f85fc06de10 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fa59c700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c5c0 tx=0x7f85f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011590 con 0x7f85fc06de10 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc16abd0 con 0x7f85fc06de10 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.383+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc16b190 con 0x7f85fc06de10 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f0011710 con 0x7f85fc06de10 2023-10-06T11:23:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011ad0 con 0x7f85fc06de10 2023-10-06T11:23:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e00052b0 con 0x7f85fc06de10 2023-10-06T11:23:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f85f001b3b0 con 0x7f85fc06de10 2023-10-06T11:23:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85ea7fc700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 0x7f85dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.387+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f85f007bf50 con 0x7f85fc06de10 2023-10-06T11:23:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.391+0000 7f85f9d9b700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 0x7f85dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.391+0000 7f85f9d9b700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 0x7f85dc062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85fc069410 tx=0x7f85ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.395+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f85f004c200 con 0x7f85fc06de10 2023-10-06T11:23:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:05.599+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f85e0005ae0 con 0x7f85fc06de10 2023-10-06T11:23:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.583+0000 7f85ea7fc700 1 -- 172.21.15.7:0/1880910220 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85f0017020 con 0x7f85fc06de10 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 msgr2=0x7f85dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 0x7f85dc062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85fc069410 tx=0x7f85ec008040 comp rx=0 tx=0).stop 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 msgr2=0x7f85fc16ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c5c0 tx=0x7f85f000c8d0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 shutdown_connections 2023-10-06T11:23:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f85dc060550 0x7f85dc062a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85fc0b41e0 0x7f85fc169810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f85fc0b3e10 0x7f85fc16d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 --2- 172.21.15.7:0/1880910220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc06de10 0x7f85fc16ce50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 >> 172.21.15.7:0/1880910220 conn(0x7f85fc0cfd00 msgr2=0x7f85fc14f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 shutdown_connections 2023-10-06T11:23:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.587+0000 7f85fb59e700 1 -- 172.21.15.7:0/1880910220 wait complete. 2023-10-06T11:23:06.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:06.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.711+0000 7fca04f11700 1 Processor -- start 2023-10-06T11:23:06.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.711+0000 7fca04f11700 1 -- start start 2023-10-06T11:23:06.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca0006a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca00157060 0x7fca00156340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 0x7fca00156880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca00156dc0 con 0x7fca00157060 2023-10-06T11:23:06.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca00155eb0 con 0x7fca001574b0 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fca04f11700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0006a870 con 0x7fca00155630 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca00157060 0x7fca00156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca00157060 0x7fca00156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54136/0 (socket says 172.21.15.7:54136) 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/3538869935 learned_addr learned my addr 172.21.15.7:0/3538869935 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fed9d700 1 --2- 172.21.15.7:0/3538869935 >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 0x7fca00156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fe59c700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca0006a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/3538869935 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 msgr2=0x7fca0006a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca0006a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/3538869935 >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 msgr2=0x7fca00156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/3538869935 >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 0x7fca00156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/3538869935 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0006ab50 con 0x7fca00157060 2023-10-06T11:23:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fed9d700 1 --2- 172.21.15.7:0/3538869935 >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 0x7fca00156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:06.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca00156340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca0006e030 tx=0x7fc9e800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de20967ae290b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fdd9b700 1 -- 172.21.15.7:0/3538869935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800c970 con 0x7fca00157060 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fdd9b700 1 -- 172.21.15.7:0/3538869935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800cad0 con 0x7fca00157060 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.715+0000 7fc9fdd9b700 1 -- 172.21.15.7:0/3538869935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800ce70 con 0x7fca00157060 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 -- 172.21.15.7:0/3538869935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 msgr2=0x7fca00156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca00156340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca0006e030 tx=0x7fc9e800a3f0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 -- 172.21.15.7:0/3538869935 shutdown_connections 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 --2- 172.21.15.7:0/3538869935 >> v2:172.21.15.50:3300/0 conn(0x7fca001574b0 0x7fca00156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca00156340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 --2- 172.21.15.7:0/3538869935 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca0006a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 -- 172.21.15.7:0/3538869935 >> 172.21.15.7:0/3538869935 conn(0x7fca00067e30 msgr2=0x7fca00068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:06.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 -- 172.21.15.7:0/3538869935 shutdown_connections 2023-10-06T11:23:06.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.719+0000 7fca04f11700 1 -- 172.21.15.7:0/3538869935 wait complete. 2023-10-06T11:23:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 Processor -- start 2023-10-06T11:23:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- start start 2023-10-06T11:23:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca00169240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 0x7fca000a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca00146730 con 0x7fca00157060 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca00145b20 con 0x7fca001574b0 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca00145e00 con 0x7fca00155630 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54138/0 (socket says 172.21.15.7:54138) 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/1854388182 learned_addr learned my addr 172.21.15.7:0/1854388182 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9fed9d700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 0x7fca000a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9fe59c700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca00169240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 msgr2=0x7fca00169240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca00169240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 msgr2=0x7fca000a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 0x7fca000a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 -- 172.21.15.7:0/1854388182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8009a20 con 0x7fca00157060 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9fed9d700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 0x7fca000a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f5d9b700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca00064280 tx=0x7fc9e801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8019e50 con 0x7fca00157060 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8010600 con 0x7fca00157060 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e800a690 con 0x7fca00157060 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80109a0 con 0x7fca00157060 2023-10-06T11:23:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca00169780 con 0x7fca00157060 2023-10-06T11:23:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.723+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca001468e0 con 0x7fca00157060 2023-10-06T11:23:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.727+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc9e802a3b0 con 0x7fca00157060 2023-10-06T11:23:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.727+0000 7fc9f6ffd700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 0x7fc9e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.727+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc9e807e2a0 con 0x7fca00157060 2023-10-06T11:23:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.727+0000 7fc9fe59c700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 0x7fc9e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.727+0000 7fc9fe59c700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 0x7fc9e0062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009ca0 tx=0x7fc9f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.731+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc9e804e670 con 0x7fca00157060 2023-10-06T11:23:06.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.923+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca00146870 con 0x7fca00157060 2023-10-06T11:23:06.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fc9f6ffd700 1 -- 172.21.15.7:0/1854388182 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7fc9e8051c90 con 0x7fca00157060 2023-10-06T11:23:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:06.930 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:06.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 msgr2=0x7fc9e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 0x7fc9e0062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009ca0 tx=0x7fc9f0008040 comp rx=0 tx=0).stop 2023-10-06T11:23:06.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 msgr2=0x7fca001451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca00064280 tx=0x7fc9e801a000 comp rx=0 tx=0).stop 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 shutdown_connections 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc9e0060550 0x7fc9e0062a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca001574b0 0x7fca000a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca00157060 0x7fca001451d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 --2- 172.21.15.7:0/1854388182 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fca00155630 0x7fca00169240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 >> 172.21.15.7:0/1854388182 conn(0x7fca00067e30 msgr2=0x7fca0006e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 shutdown_connections 2023-10-06T11:23:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:06.927+0000 7fca04f11700 1 -- 172.21.15.7:0/1854388182 wait complete. 2023-10-06T11:23:06.934 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:06.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.047+0000 7f0d9e3c9700 1 Processor -- start 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- start start 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d980abf70 0x7f0d98159820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c9020 con 0x7f0d980abba0 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c8770 con 0x7f0d98159d60 2023-10-06T11:23:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d980cd510 con 0x7f0d980abf70 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d980abf70 0x7f0d98159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d97fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d980abba0 0x7f0d980cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49726/0 (socket says 172.21.15.7:49726) 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d97fff700 1 -- 172.21.15.7:0/3543564117 learned_addr learned my addr 172.21.15.7:0/3543564117 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d977fe700 1 -- 172.21.15.7:0/3543564117 >> v1:172.21.15.50:6789/0 conn(0x7f0d98159d60 legacy=0x7f0d980c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d977fe700 1 -- 172.21.15.7:0/3543564117 >> v1:172.21.15.7:6789/0 conn(0x7f0d980abba0 legacy=0x7f0d980cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d977fe700 1 -- 172.21.15.7:0/3543564117 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980c8a10 con 0x7f0d980abf70 2023-10-06T11:23:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d977fe700 1 --2- 172.21.15.7:0/3543564117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abf70 0x7f0d98159820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098e0 tx=0x7f0d800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550e5670ec622d82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d96ffd700 1 -- 172.21.15.7:0/3543564117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80009e80 con 0x7f0d980abf70 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d96ffd700 1 -- 172.21.15.7:0/3543564117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000b860 con 0x7f0d980abf70 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d96ffd700 1 -- 172.21.15.7:0/3543564117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000bbe0 con 0x7f0d980abf70 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/3543564117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abf70 msgr2=0x7f0d98159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/3543564117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abf70 0x7f0d98159820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098e0 tx=0x7f0d800091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/3543564117 shutdown_connections 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/3543564117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abf70 0x7f0d98159820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/3543564117 >> 172.21.15.7:0/3543564117 conn(0x7f0d9806b8a0 msgr2=0x7f0d9806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/3543564117 shutdown_connections 2023-10-06T11:23:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.051+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/3543564117 wait complete. 2023-10-06T11:23:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 Processor -- start 2023-10-06T11:23:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 -- start start 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0d980abf70 0x7f0d981583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d98159d60 0x7f0d981588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98149b80 con 0x7f0d980abba0 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980acda0 con 0x7f0d98159d60 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98158e30 con 0x7f0d980abf70 2023-10-06T11:23:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54166/0 (socket says 172.21.15.7:54166) 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 -- 172.21.15.7:0/691752010 learned_addr learned my addr 172.21.15.7:0/691752010 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9c966700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d98159d60 0x7f0d981588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d977fe700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0d980abf70 0x7f0d981583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 -- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0d980abf70 msgr2=0x7f0d981583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0d980abf70 0x7f0d981583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 -- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d98159d60 msgr2=0x7f0d981588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d98159d60 0x7f0d981588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 -- 172.21.15.7:0/691752010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009420 con 0x7f0d980abba0 2023-10-06T11:23:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d97fff700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800c7c0 tx=0x7f0d88007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88005b30 con 0x7f0d980abba0 2023-10-06T11:23:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88005c90 con 0x7f0d980abba0 2023-10-06T11:23:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d980ac4b0 con 0x7f0d980abba0 2023-10-06T11:23:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.055+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88004a80 con 0x7f0d980abba0 2023-10-06T11:23:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d980acab0 con 0x7f0d980abba0 2023-10-06T11:23:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0d880070d0 con 0x7f0d980abba0 2023-10-06T11:23:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98159180 con 0x7f0d980abba0 2023-10-06T11:23:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d8ffff700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 0x7f0d78062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0d8807bd60 con 0x7f0d980abba0 2023-10-06T11:23:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.059+0000 7f0d977fe700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 0x7f0d78062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.063+0000 7f0d977fe700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 0x7f0d78062a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098e0 tx=0x7f0d80019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.067+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d88049be0 con 0x7f0d980abba0 2023-10-06T11:23:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.235+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d9804be00 con 0x7f0d980abba0 2023-10-06T11:23:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.235+0000 7f0d8ffff700 1 -- 172.21.15.7:0/691752010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16975 (secure 0 0 0) 0x7f0d98159180 con 0x7f0d980abba0 2023-10-06T11:23:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:07.241 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:04.557305+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 msgr2=0x7f0d78062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 0x7f0d78062a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098e0 tx=0x7f0d80019000 comp rx=0 tx=0).stop 2023-10-06T11:23:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 msgr2=0x7f0d98156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800c7c0 tx=0x7f0d88007f70 comp rx=0 tx=0).stop 2023-10-06T11:23:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 shutdown_connections 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0d78060580 0x7f0d78062a30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d98159d60 0x7f0d981588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0d980abf70 0x7f0d981583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 --2- 172.21.15.7:0/691752010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d980abba0 0x7f0d98156540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 >> 172.21.15.7:0/691752010 conn(0x7f0d9806b8a0 msgr2=0x7f0d980cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 shutdown_connections 2023-10-06T11:23:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.239+0000 7f0d9e3c9700 1 -- 172.21.15.7:0/691752010 wait complete. 2023-10-06T11:23:07.257 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:23:07.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 Processor -- start 2023-10-06T11:23:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 -- start start 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 0x7fe1040af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10415b3c0 con 0x7fe10415ab80 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10415f540 con 0x7fe1040af8c0 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe10a8c3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040afd40 con 0x7fe10415af50 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1098c1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe10415ab80 0x7fe1040ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49736/0 (socket says 172.21.15.7:49736) 2023-10-06T11:23:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1098c1700 1 -- 172.21.15.7:0/677720416 learned_addr learned my addr 172.21.15.7:0/677720416 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1090c0700 1 --2- 172.21.15.7:0/677720416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 0x7fe1040af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213164472 0 0) 0x7fe10415b3c0 con 0x7fe10415ab80 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0003570 con 0x7fe10415ab80 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895423378 0 0) 0x7fe10415f540 con 0x7fe1040af8c0 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10415b3c0 con 0x7fe1040af8c0 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1090c0700 1 -- 172.21.15.7:0/677720416 >> v1:172.21.15.50:6789/0 conn(0x7fe1040af8c0 legacy=0x7fe10415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1090c0700 1 -- 172.21.15.7:0/677720416 >> v1:172.21.15.7:6789/0 conn(0x7fe10415ab80 legacy=0x7fe1040ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4168468676 0 0) 0x7fe0f0003570 con 0x7fe10415ab80 2023-10-06T11:23:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.363+0000 7fe1090c0700 1 -- 172.21.15.7:0/677720416 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10415e950 con 0x7fe10415af50 2023-10-06T11:23:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe1090c0700 1 --2- 172.21.15.7:0/677720416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 0x7fe1040af380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009910 tx=0x7fe0f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64328ec2d5ae04b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0017f0 con 0x7fe10415af50 2023-10-06T11:23:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f4011420 con 0x7fe10415af50 2023-10-06T11:23:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe1088bf700 1 -- 172.21.15.7:0/677720416 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40117a0 con 0x7fe10415af50 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 -- 172.21.15.7:0/677720416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 msgr2=0x7fe1040af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/677720416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 0x7fe1040af380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009910 tx=0x7fe0f40091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 -- 172.21.15.7:0/677720416 shutdown_connections 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/677720416 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe10415af50 0x7fe1040af380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 -- 172.21.15.7:0/677720416 >> 172.21.15.7:0/677720416 conn(0x7fe1040688a0 msgr2=0x7fe1040cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 -- 172.21.15.7:0/677720416 shutdown_connections 2023-10-06T11:23:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.367+0000 7fe10a8c3700 1 -- 172.21.15.7:0/677720416 wait complete. 2023-10-06T11:23:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 Processor -- start 2023-10-06T11:23:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 -- start start 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1040af8c0 0x7fe10416e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe10415ab80 0x7fe1040adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040c5df0 con 0x7fe10415af50 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040c5960 con 0x7fe10415ab80 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040ae1a0 con 0x7fe1040af8c0 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54182/0 (socket says 172.21.15.7:54182) 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 -- 172.21.15.7:0/1002794901 learned_addr learned my addr 172.21.15.7:0/1002794901 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe1090c0700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe10415ab80 0x7fe1040adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe1098c1700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1040af8c0 0x7fe10416e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 -- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1040af8c0 msgr2=0x7fe10416e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1040af8c0 0x7fe10416e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 -- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe10415ab80 msgr2=0x7fe1040adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe10415ab80 0x7fe1040adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 -- 172.21.15.7:0/1002794901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009420 con 0x7fe10415af50 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a0c2700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c590 tx=0x7fe100007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100016070 con 0x7fe10415af50 2023-10-06T11:23:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100011590 con 0x7fe10415af50 2023-10-06T11:23:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1040c4d50 con 0x7fe10415af50 2023-10-06T11:23:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.371+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100011970 con 0x7fe10415af50 2023-10-06T11:23:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1040c5310 con 0x7fe10415af50 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10406c710 con 0x7fe10415af50 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe100012420 con 0x7fe10415af50 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe0f9ffb700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 0x7fe0e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe10007c6d0 con 0x7fe10415af50 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe1098c1700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 0x7fe0e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.375+0000 7fe1098c1700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 0x7fe0e8062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc000f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.383+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe10004ca20 con 0x7fe10415af50 2023-10-06T11:23:07.526 INFO:tasks.ceph.mds.a.smithi007.stderr:2023-10-06T11:23:07.523+0000 7f554cdf6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:23:07.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.547+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe10415ecb0 con 0x7fe0e8060550 2023-10-06T11:23:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.563+0000 7fe0f9ffb700 1 -- 172.21.15.7:0/1002794901 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309525 (secure 0 0 0) 0x7fe0fc001e30 con 0x7fe0e8060550 2023-10-06T11:23:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:07.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 msgr2=0x7fe0e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 0x7fe0e8062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc000f40 comp rx=0 tx=0).stop 2023-10-06T11:23:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 msgr2=0x7fe1040c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c590 tx=0x7fe100007d80 comp rx=0 tx=0).stop 2023-10-06T11:23:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 shutdown_connections 2023-10-06T11:23:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe0e8060550 0x7fe0e8062a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe10415af50 0x7fe1040c6950 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe10415ab80 0x7fe1040adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 --2- 172.21.15.7:0/1002794901 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe1040af8c0 0x7fe10416e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 >> 172.21.15.7:0/1002794901 conn(0x7fe1040688a0 msgr2=0x7fe1040ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 shutdown_connections 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.571+0000 7fe10a8c3700 1 -- 172.21.15.7:0/1002794901 wait complete. 2023-10-06T11:23:07.577 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:07.588 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-10-06T11:23:05.836732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415756,"kb_used_data":4684,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752558964,"statfs":{"total":773094113280,"available":770620379136,"internally_reserved":0,"allocated":4796416,"data_stored":2766567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.292603"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587558+0000","last_change":"2023-10-06T11:23:03.587558+0000","last_active":"2023-10-06T11:23:03.587558+0000","last_peered":"2023-10-06T11:23:03.587558+0000","last_clean":"2023-10-06T11:23:03.587558+0000","last_became_active":"2023-10-06T11:23:03.584983+0000","last_became_peered":"2023-10-06T11:23:03.584983+0000","last_unstale":"2023-10-06T11:23:03.587558+0000","last_undegraded":"2023-10-06T11:23:03.587558+0000","last_fullsized":"2023-10-06T11:23:03.587558+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593143+0000","last_change":"2023-10-06T11:23:03.593143+0000","last_active":"2023-10-06T11:23:03.593143+0000","last_peered":"2023-10-06T11:23:03.593143+0000","last_clean":"2023-10-06T11:23:03.593143+0000","last_became_active":"2023-10-06T11:23:03.592544+0000","last_became_peered":"2023-10-06T11:23:03.592544+0000","last_unstale":"2023-10-06T11:23:03.593143+0000","last_undegraded":"2023-10-06T11:23:03.593143+0000","last_fullsized":"2023-10-06T11:23:03.593143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:05:24.926443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585950+0000","last_change":"2023-10-06T11:23:03.582956+0000","last_active":"2023-10-06T11:23:03.585950+0000","last_peered":"2023-10-06T11:23:03.585950+0000","last_clean":"2023-10-06T11:23:03.585950+0000","last_became_active":"2023-10-06T11:23:03.582628+0000","last_became_peered":"2023-10-06T11:23:03.582628+0000","last_unstale":"2023-10-06T11:23:03.585950+0000","last_undegraded":"2023-10-06T11:23:03.585950+0000","last_fullsized":"2023-10-06T11:23:03.585950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591639+0000","last_change":"2023-10-06T11:23:03.591639+0000","last_active":"2023-10-06T11:23:03.591639+0000","last_peered":"2023-10-06T11:23:03.591639+0000","last_clean":"2023-10-06T11:23:03.591639+0000","last_became_active":"2023-10-06T11:23:03.591149+0000","last_became_peered":"2023-10-06T11:23:03.591149+0000","last_unstale":"2023-10-06T11:23:03.591639+0000","last_undegraded":"2023-10-06T11:23:03.591639+0000","last_fullsized":"2023-10-06T11:23:03.591639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:10.120082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592151+0000","last_change":"2023-10-06T11:23:03.592151+0000","last_active":"2023-10-06T11:23:03.592151+0000","last_peered":"2023-10-06T11:23:03.592151+0000","last_clean":"2023-10-06T11:23:03.592151+0000","last_became_active":"2023-10-06T11:23:03.591779+0000","last_became_peered":"2023-10-06T11:23:03.591779+0000","last_unstale":"2023-10-06T11:23:03.592151+0000","last_undegraded":"2023-10-06T11:23:03.592151+0000","last_fullsized":"2023-10-06T11:23:03.592151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.818097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581242+0000","last_change":"2023-10-06T11:23:03.578183+0000","last_active":"2023-10-06T11:23:03.581242+0000","last_peered":"2023-10-06T11:23:03.581242+0000","last_clean":"2023-10-06T11:23:03.581242+0000","last_became_active":"2023-10-06T11:23:03.577805+0000","last_became_peered":"2023-10-06T11:23:03.577805+0000","last_unstale":"2023-10-06T11:23:03.581242+0000","last_undegraded":"2023-10-06T11:23:03.581242+0000","last_fullsized":"2023-10-06T11:23:03.581242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:28.588171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584028+0000","last_change":"2023-10-06T11:23:03.584028+0000","last_active":"2023-10-06T11:23:03.584028+0000","last_peered":"2023-10-06T11:23:03.584028+0000","last_clean":"2023-10-06T11:23:03.584028+0000","last_became_active":"2023-10-06T11:23:03.583722+0000","last_became_peered":"2023-10-06T11:23:03.583722+0000","last_unstale":"2023-10-06T11:23:03.584028+0000","last_undegraded":"2023-10-06T11:23:03.584028+0000","last_fullsized":"2023-10-06T11:23:03.584028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589584+0000","last_change":"2023-10-06T11:23:03.589584+0000","last_active":"2023-10-06T11:23:03.589584+0000","last_peered":"2023-10-06T11:23:03.589584+0000","last_clean":"2023-10-06T11:23:03.589584+0000","last_became_active":"2023-10-06T11:23:03.588153+0000","last_became_peered":"2023-10-06T11:23:03.588153+0000","last_unstale":"2023-10-06T11:23:03.589584+0000","last_undegraded":"2023-10-06T11:23:03.589584+0000","last_fullsized":"2023-10-06T11:23:03.589584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598463+0000","last_change":"2023-10-06T11:23:03.598463+0000","last_active":"2023-10-06T11:23:03.598463+0000","last_peered":"2023-10-06T11:23:03.598463+0000","last_clean":"2023-10-06T11:23:03.598463+0000","last_became_active":"2023-10-06T11:23:03.598120+0000","last_became_peered":"2023-10-06T11:23:03.598120+0000","last_unstale":"2023-10-06T11:23:03.598463+0000","last_undegraded":"2023-10-06T11:23:03.598463+0000","last_fullsized":"2023-10-06T11:23:03.598463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:18.747941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599961+0000","last_change":"2023-10-06T11:23:03.599961+0000","last_active":"2023-10-06T11:23:03.599961+0000","last_peered":"2023-10-06T11:23:03.599961+0000","last_clean":"2023-10-06T11:23:03.599961+0000","last_became_active":"2023-10-06T11:23:03.599618+0000","last_became_peered":"2023-10-06T11:23:03.599618+0000","last_unstale":"2023-10-06T11:23:03.599961+0000","last_undegraded":"2023-10-06T11:23:03.599961+0000","last_fullsized":"2023-10-06T11:23:03.599961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:19.535711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598084+0000","last_change":"2023-10-06T11:23:03.598084+0000","last_active":"2023-10-06T11:23:03.598084+0000","last_peered":"2023-10-06T11:23:03.598084+0000","last_clean":"2023-10-06T11:23:03.598084+0000","last_became_active":"2023-10-06T11:23:03.597694+0000","last_became_peered":"2023-10-06T11:23:03.597694+0000","last_unstale":"2023-10-06T11:23:03.598084+0000","last_undegraded":"2023-10-06T11:23:03.598084+0000","last_fullsized":"2023-10-06T11:23:03.598084+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:35.666497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585891+0000","last_change":"2023-10-06T11:23:03.585891+0000","last_active":"2023-10-06T11:23:03.585891+0000","last_peered":"2023-10-06T11:23:03.585891+0000","last_clean":"2023-10-06T11:23:03.585891+0000","last_became_active":"2023-10-06T11:23:03.585297+0000","last_became_peered":"2023-10-06T11:23:03.585297+0000","last_unstale":"2023-10-06T11:23:03.585891+0000","last_undegraded":"2023-10-06T11:23:03.585891+0000","last_fullsized":"2023-10-06T11:23:03.585891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:14.203173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581685+0000","last_change":"2023-10-06T11:23:03.580379+0000","last_active":"2023-10-06T11:23:03.581685+0000","last_peered":"2023-10-06T11:23:03.581685+0000","last_clean":"2023-10-06T11:23:03.581685+0000","last_became_active":"2023-10-06T11:23:03.580083+0000","last_became_peered":"2023-10-06T11:23:03.580083+0000","last_unstale":"2023-10-06T11:23:03.581685+0000","last_undegraded":"2023-10-06T11:23:03.581685+0000","last_fullsized":"2023-10-06T11:23:03.581685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:28:14.475031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.597730+0000","last_change":"2023-10-06T11:23:03.597730+0000","last_active":"2023-10-06T11:23:03.597730+0000","last_peered":"2023-10-06T11:23:03.597730+0000","last_clean":"2023-10-06T11:23:03.597730+0000","last_became_active":"2023-10-06T11:23:03.597423+0000","last_became_peered":"2023-10-06T11:23:03.597423+0000","last_unstale":"2023-10-06T11:23:03.597730+0000","last_undegraded":"2023-10-06T11:23:03.597730+0000","last_fullsized":"2023-10-06T11:23:03.597730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585328+0000","last_change":"2023-10-06T11:23:03.583822+0000","last_active":"2023-10-06T11:23:03.585328+0000","last_peered":"2023-10-06T11:23:03.585328+0000","last_clean":"2023-10-06T11:23:03.585328+0000","last_became_active":"2023-10-06T11:23:03.583571+0000","last_became_peered":"2023-10-06T11:23:03.583571+0000","last_unstale":"2023-10-06T11:23:03.585328+0000","last_undegraded":"2023-10-06T11:23:03.585328+0000","last_fullsized":"2023-10-06T11:23:03.585328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593956+0000","last_change":"2023-10-06T11:23:03.591839+0000","last_active":"2023-10-06T11:23:03.593956+0000","last_peered":"2023-10-06T11:23:03.593956+0000","last_clean":"2023-10-06T11:23:03.593956+0000","last_became_active":"2023-10-06T11:23:03.591528+0000","last_became_peered":"2023-10-06T11:23:03.591528+0000","last_unstale":"2023-10-06T11:23:03.593956+0000","last_undegraded":"2023-10-06T11:23:03.593956+0000","last_fullsized":"2023-10-06T11:23:03.593956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:41.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585798+0000","last_change":"2023-10-06T11:23:03.582937+0000","last_active":"2023-10-06T11:23:03.585798+0000","last_peered":"2023-10-06T11:23:03.585798+0000","last_clean":"2023-10-06T11:23:03.585798+0000","last_became_active":"2023-10-06T11:23:03.582598+0000","last_became_peered":"2023-10-06T11:23:03.582598+0000","last_unstale":"2023-10-06T11:23:03.585798+0000","last_undegraded":"2023-10-06T11:23:03.585798+0000","last_fullsized":"2023-10-06T11:23:03.585798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:44.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.594417+0000","last_change":"2023-10-06T11:23:03.592964+0000","last_active":"2023-10-06T11:23:03.594417+0000","last_peered":"2023-10-06T11:23:03.594417+0000","last_clean":"2023-10-06T11:23:03.594417+0000","last_became_active":"2023-10-06T11:23:03.592611+0000","last_became_peered":"2023-10-06T11:23:03.592611+0000","last_unstale":"2023-10-06T11:23:03.594417+0000","last_undegraded":"2023-10-06T11:23:03.594417+0000","last_fullsized":"2023-10-06T11:23:03.594417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:41.947324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.601395+0000","last_change":"2023-10-06T11:23:03.599987+0000","last_active":"2023-10-06T11:23:03.601395+0000","last_peered":"2023-10-06T11:23:03.601395+0000","last_clean":"2023-10-06T11:23:03.601395+0000","last_became_active":"2023-10-06T11:23:03.599658+0000","last_became_peered":"2023-10-06T11:23:03.599658+0000","last_unstale":"2023-10-06T11:23:03.601395+0000","last_undegraded":"2023-10-06T11:23:03.601395+0000","last_fullsized":"2023-10-06T11:23:03.601395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:57.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589149+0000","last_change":"2023-10-06T11:23:03.587617+0000","last_active":"2023-10-06T11:23:03.589149+0000","last_peered":"2023-10-06T11:23:03.589149+0000","last_clean":"2023-10-06T11:23:03.589149+0000","last_became_active":"2023-10-06T11:23:03.587337+0000","last_became_peered":"2023-10-06T11:23:03.587337+0000","last_unstale":"2023-10-06T11:23:03.589149+0000","last_undegraded":"2023-10-06T11:23:03.589149+0000","last_fullsized":"2023-10-06T11:23:03.589149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:02.611041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574406+0000","last_change":"2023-10-06T11:23:03.570300+0000","last_active":"2023-10-06T11:23:03.574406+0000","last_peered":"2023-10-06T11:23:03.574406+0000","last_clean":"2023-10-06T11:23:03.574406+0000","last_became_active":"2023-10-06T11:23:03.569859+0000","last_became_peered":"2023-10-06T11:23:03.569859+0000","last_unstale":"2023-10-06T11:23:03.574406+0000","last_undegraded":"2023-10-06T11:23:03.574406+0000","last_fullsized":"2023-10-06T11:23:03.574406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:33.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587337+0000","last_change":"2023-10-06T11:23:03.587337+0000","last_active":"2023-10-06T11:23:03.587337+0000","last_peered":"2023-10-06T11:23:03.587337+0000","last_clean":"2023-10-06T11:23:03.587337+0000","last_became_active":"2023-10-06T11:23:03.586816+0000","last_became_peered":"2023-10-06T11:23:03.586816+0000","last_unstale":"2023-10-06T11:23:03.587337+0000","last_undegraded":"2023-10-06T11:23:03.587337+0000","last_fullsized":"2023-10-06T11:23:03.587337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:07.586839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581935+0000","last_change":"2023-10-06T11:23:03.579258+0000","last_active":"2023-10-06T11:23:03.581935+0000","last_peered":"2023-10-06T11:23:03.581935+0000","last_clean":"2023-10-06T11:23:03.581935+0000","last_became_active":"2023-10-06T11:23:03.578821+0000","last_became_peered":"2023-10-06T11:23:03.578821+0000","last_unstale":"2023-10-06T11:23:03.581935+0000","last_undegraded":"2023-10-06T11:23:03.581935+0000","last_fullsized":"2023-10-06T11:23:03.581935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:26.243118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.580861+0000","last_change":"2023-10-06T11:23:03.578540+0000","last_active":"2023-10-06T11:23:03.580861+0000","last_peered":"2023-10-06T11:23:03.580861+0000","last_clean":"2023-10-06T11:23:03.580861+0000","last_became_active":"2023-10-06T11:23:03.578194+0000","last_became_peered":"2023-10-06T11:23:03.578194+0000","last_unstale":"2023-10-06T11:23:03.580861+0000","last_undegraded":"2023-10-06T11:23:03.580861+0000","last_fullsized":"2023-10-06T11:23:03.580861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:36:40.702516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592136+0000","last_change":"2023-10-06T11:23:03.582503+0000","last_active":"2023-10-06T11:23:03.592136+0000","last_peered":"2023-10-06T11:23:03.592136+0000","last_clean":"2023-10-06T11:23:03.592136+0000","last_became_active":"2023-10-06T11:23:03.582190+0000","last_became_peered":"2023-10-06T11:23:03.582190+0000","last_unstale":"2023-10-06T11:23:03.592136+0000","last_undegraded":"2023-10-06T11:23:03.592136+0000","last_fullsized":"2023-10-06T11:23:03.592136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:40:59.081090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574567+0000","last_change":"2023-10-06T11:23:03.572272+0000","last_active":"2023-10-06T11:23:03.574567+0000","last_peered":"2023-10-06T11:23:03.574567+0000","last_clean":"2023-10-06T11:23:03.574567+0000","last_became_active":"2023-10-06T11:23:03.571933+0000","last_became_peered":"2023-10-06T11:23:03.571933+0000","last_unstale":"2023-10-06T11:23:03.574567+0000","last_undegraded":"2023-10-06T11:23:03.574567+0000","last_fullsized":"2023-10-06T11:23:03.574567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:36:40.702516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585973+0000","last_change":"2023-10-06T11:23:03.583302+0000","last_active":"2023-10-06T11:23:03.585973+0000","last_peered":"2023-10-06T11:23:03.585973+0000","last_clean":"2023-10-06T11:23:03.585973+0000","last_became_active":"2023-10-06T11:23:03.582559+0000","last_became_peered":"2023-10-06T11:23:03.582559+0000","last_unstale":"2023-10-06T11:23:03.585973+0000","last_undegraded":"2023-10-06T11:23:03.585973+0000","last_fullsized":"2023-10-06T11:23:03.585973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:28.588171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585646+0000","last_change":"2023-10-06T11:23:03.585646+0000","last_active":"2023-10-06T11:23:03.585646+0000","last_peered":"2023-10-06T11:23:03.585646+0000","last_clean":"2023-10-06T11:23:03.585646+0000","last_became_active":"2023-10-06T11:23:03.584862+0000","last_became_peered":"2023-10-06T11:23:03.584862+0000","last_unstale":"2023-10-06T11:23:03.585646+0000","last_undegraded":"2023-10-06T11:23:03.585646+0000","last_fullsized":"2023-10-06T11:23:03.585646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593409+0000","last_change":"2023-10-06T11:23:03.586648+0000","last_active":"2023-10-06T11:23:03.593409+0000","last_peered":"2023-10-06T11:23:03.593409+0000","last_clean":"2023-10-06T11:23:03.593409+0000","last_became_active":"2023-10-06T11:23:03.586331+0000","last_became_peered":"2023-10-06T11:23:03.586331+0000","last_unstale":"2023-10-06T11:23:03.593409+0000","last_undegraded":"2023-10-06T11:23:03.593409+0000","last_fullsized":"2023-10-06T11:23:03.593409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:27.511696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591147+0000","last_change":"2023-10-06T11:23:03.588489+0000","last_active":"2023-10-06T11:23:03.591147+0000","last_peered":"2023-10-06T11:23:03.591147+0000","last_clean":"2023-10-06T11:23:03.591147+0000","last_became_active":"2023-10-06T11:23:03.588187+0000","last_became_peered":"2023-10-06T11:23:03.588187+0000","last_unstale":"2023-10-06T11:23:03.591147+0000","last_undegraded":"2023-10-06T11:23:03.591147+0000","last_fullsized":"2023-10-06T11:23:03.591147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:41.504246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581164+0000","last_change":"2023-10-06T11:23:03.576827+0000","last_active":"2023-10-06T11:23:03.581164+0000","last_peered":"2023-10-06T11:23:03.581164+0000","last_clean":"2023-10-06T11:23:03.581164+0000","last_became_active":"2023-10-06T11:23:03.576480+0000","last_became_peered":"2023-10-06T11:23:03.576480+0000","last_unstale":"2023-10-06T11:23:03.581164+0000","last_undegraded":"2023-10-06T11:23:03.581164+0000","last_fullsized":"2023-10-06T11:23:03.581164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:03.914121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.588818+0000","last_change":"2023-10-06T11:23:03.586173+0000","last_active":"2023-10-06T11:23:03.588818+0000","last_peered":"2023-10-06T11:23:03.588818+0000","last_clean":"2023-10-06T11:23:03.588818+0000","last_became_active":"2023-10-06T11:23:03.585842+0000","last_became_peered":"2023-10-06T11:23:03.585842+0000","last_unstale":"2023-10-06T11:23:03.588818+0000","last_undegraded":"2023-10-06T11:23:03.588818+0000","last_fullsized":"2023-10-06T11:23:03.588818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:51:14.805762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585863+0000","last_change":"2023-10-06T11:23:03.585863+0000","last_active":"2023-10-06T11:23:03.585863+0000","last_peered":"2023-10-06T11:23:03.585863+0000","last_clean":"2023-10-06T11:23:03.585863+0000","last_became_active":"2023-10-06T11:23:03.585255+0000","last_became_peered":"2023-10-06T11:23:03.585255+0000","last_unstale":"2023-10-06T11:23:03.585863+0000","last_undegraded":"2023-10-06T11:23:03.585863+0000","last_fullsized":"2023-10-06T11:23:03.585863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:41.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599269+0000","last_change":"2023-10-06T11:23:02.583346+0000","last_active":"2023-10-06T11:23:03.599269+0000","last_peered":"2023-10-06T11:23:03.599269+0000","last_clean":"2023-10-06T11:23:03.599269+0000","last_became_active":"2023-10-06T11:23:02.583017+0000","last_became_peered":"2023-10-06T11:23:02.583017+0000","last_unstale":"2023-10-06T11:23:03.599269+0000","last_undegraded":"2023-10-06T11:23:03.599269+0000","last_fullsized":"2023-10-06T11:23:03.599269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.256819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593440+0000","last_change":"2023-10-06T11:23:03.585930+0000","last_active":"2023-10-06T11:23:03.593440+0000","last_peered":"2023-10-06T11:23:03.593440+0000","last_clean":"2023-10-06T11:23:03.593440+0000","last_became_active":"2023-10-06T11:23:03.585678+0000","last_became_peered":"2023-10-06T11:23:03.585678+0000","last_unstale":"2023-10-06T11:23:03.593440+0000","last_undegraded":"2023-10-06T11:23:03.593440+0000","last_fullsized":"2023-10-06T11:23:03.593440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:17.348545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598767+0000","last_change":"2023-10-06T11:23:02.578721+0000","last_active":"2023-10-06T11:23:03.598767+0000","last_peered":"2023-10-06T11:23:03.598767+0000","last_clean":"2023-10-06T11:23:03.598767+0000","last_became_active":"2023-10-06T11:23:02.578343+0000","last_became_peered":"2023-10-06T11:23:02.578343+0000","last_unstale":"2023-10-06T11:23:03.598767+0000","last_undegraded":"2023-10-06T11:23:03.598767+0000","last_fullsized":"2023-10-06T11:23:03.598767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:33.018879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.577151+0000","last_change":"2023-10-06T11:23:03.577151+0000","last_active":"2023-10-06T11:23:03.577151+0000","last_peered":"2023-10-06T11:23:03.577151+0000","last_clean":"2023-10-06T11:23:03.577151+0000","last_became_active":"2023-10-06T11:23:03.576625+0000","last_became_peered":"2023-10-06T11:23:03.576625+0000","last_unstale":"2023-10-06T11:23:03.577151+0000","last_undegraded":"2023-10-06T11:23:03.577151+0000","last_fullsized":"2023-10-06T11:23:03.577151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:35.666497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604761+0000","last_change":"2023-10-06T11:23:02.573591+0000","last_active":"2023-10-06T11:23:03.604761+0000","last_peered":"2023-10-06T11:23:03.604761+0000","last_clean":"2023-10-06T11:23:03.604761+0000","last_became_active":"2023-10-06T11:23:02.573273+0000","last_became_peered":"2023-10-06T11:23:02.573273+0000","last_unstale":"2023-10-06T11:23:03.604761+0000","last_undegraded":"2023-10-06T11:23:03.604761+0000","last_fullsized":"2023-10-06T11:23:03.604761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.582108+0000","last_change":"2023-10-06T11:23:03.582108+0000","last_active":"2023-10-06T11:23:03.582108+0000","last_peered":"2023-10-06T11:23:03.582108+0000","last_clean":"2023-10-06T11:23:03.582108+0000","last_became_active":"2023-10-06T11:23:03.581868+0000","last_became_peered":"2023-10-06T11:23:03.581868+0000","last_unstale":"2023-10-06T11:23:03.582108+0000","last_undegraded":"2023-10-06T11:23:03.582108+0000","last_fullsized":"2023-10-06T11:23:03.582108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:39:15.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600315+0000","last_change":"2023-10-06T11:23:02.585083+0000","last_active":"2023-10-06T11:23:03.600315+0000","last_peered":"2023-10-06T11:23:03.600315+0000","last_clean":"2023-10-06T11:23:03.600315+0000","last_became_active":"2023-10-06T11:23:02.584774+0000","last_became_peered":"2023-10-06T11:23:02.584774+0000","last_unstale":"2023-10-06T11:23:03.600315+0000","last_undegraded":"2023-10-06T11:23:03.600315+0000","last_fullsized":"2023-10-06T11:23:03.600315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.262517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585806+0000","last_change":"2023-10-06T11:23:03.584786+0000","last_active":"2023-10-06T11:23:03.585806+0000","last_peered":"2023-10-06T11:23:03.585806+0000","last_clean":"2023-10-06T11:23:03.585806+0000","last_became_active":"2023-10-06T11:23:03.584463+0000","last_became_peered":"2023-10-06T11:23:03.584463+0000","last_unstale":"2023-10-06T11:23:03.585806+0000","last_undegraded":"2023-10-06T11:23:03.585806+0000","last_fullsized":"2023-10-06T11:23:03.585806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:06.123004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600185+0000","last_change":"2023-10-06T11:23:02.574898+0000","last_active":"2023-10-06T11:23:03.600185+0000","last_peered":"2023-10-06T11:23:03.600185+0000","last_clean":"2023-10-06T11:23:03.600185+0000","last_became_active":"2023-10-06T11:23:02.574567+0000","last_became_peered":"2023-10-06T11:23:02.574567+0000","last_unstale":"2023-10-06T11:23:03.600185+0000","last_undegraded":"2023-10-06T11:23:03.600185+0000","last_fullsized":"2023-10-06T11:23:03.600185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:34.654820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589456+0000","last_change":"2023-10-06T11:23:03.589456+0000","last_active":"2023-10-06T11:23:03.589456+0000","last_peered":"2023-10-06T11:23:03.589456+0000","last_clean":"2023-10-06T11:23:03.589456+0000","last_became_active":"2023-10-06T11:23:03.587069+0000","last_became_peered":"2023-10-06T11:23:03.587069+0000","last_unstale":"2023-10-06T11:23:03.589456+0000","last_undegraded":"2023-10-06T11:23:03.589456+0000","last_fullsized":"2023-10-06T11:23:03.589456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:03.914121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600305+0000","last_change":"2023-10-06T11:23:02.571475+0000","last_active":"2023-10-06T11:23:03.600305+0000","last_peered":"2023-10-06T11:23:03.600305+0000","last_clean":"2023-10-06T11:23:03.600305+0000","last_became_active":"2023-10-06T11:23:02.571157+0000","last_became_peered":"2023-10-06T11:23:02.571157+0000","last_unstale":"2023-10-06T11:23:03.600305+0000","last_undegraded":"2023-10-06T11:23:03.600305+0000","last_fullsized":"2023-10-06T11:23:03.600305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:06.575163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589987+0000","last_change":"2023-10-06T11:23:03.589987+0000","last_active":"2023-10-06T11:23:03.589987+0000","last_peered":"2023-10-06T11:23:03.589987+0000","last_clean":"2023-10-06T11:23:03.589987+0000","last_became_active":"2023-10-06T11:23:03.589361+0000","last_became_peered":"2023-10-06T11:23:03.589361+0000","last_unstale":"2023-10-06T11:23:03.589987+0000","last_undegraded":"2023-10-06T11:23:03.589987+0000","last_fullsized":"2023-10-06T11:23:03.589987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:33.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585757+0000","last_change":"2023-10-06T11:23:02.583464+0000","last_active":"2023-10-06T11:23:02.585757+0000","last_peered":"2023-10-06T11:23:02.585757+0000","last_clean":"2023-10-06T11:23:02.585757+0000","last_became_active":"2023-10-06T11:23:02.582470+0000","last_became_peered":"2023-10-06T11:23:02.582470+0000","last_unstale":"2023-10-06T11:23:02.585757+0000","last_undegraded":"2023-10-06T11:23:02.585757+0000","last_fullsized":"2023-10-06T11:23:02.585757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:25.231442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.582006+0000","last_change":"2023-10-06T11:23:03.579545+0000","last_active":"2023-10-06T11:23:03.582006+0000","last_peered":"2023-10-06T11:23:03.582006+0000","last_clean":"2023-10-06T11:23:03.582006+0000","last_became_active":"2023-10-06T11:23:03.579186+0000","last_became_peered":"2023-10-06T11:23:03.579186+0000","last_unstale":"2023-10-06T11:23:03.582006+0000","last_undegraded":"2023-10-06T11:23:03.582006+0000","last_fullsized":"2023-10-06T11:23:03.582006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:46.920650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586019+0000","last_change":"2023-10-06T11:23:02.585174+0000","last_active":"2023-10-06T11:23:02.586019+0000","last_peered":"2023-10-06T11:23:02.586019+0000","last_clean":"2023-10-06T11:23:02.586019+0000","last_became_active":"2023-10-06T11:23:02.583075+0000","last_became_peered":"2023-10-06T11:23:02.583075+0000","last_unstale":"2023-10-06T11:23:02.586019+0000","last_undegraded":"2023-10-06T11:23:02.586019+0000","last_fullsized":"2023-10-06T11:23:02.586019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:25.231442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.601302+0000","last_change":"2023-10-06T11:23:03.599830+0000","last_active":"2023-10-06T11:23:03.601302+0000","last_peered":"2023-10-06T11:23:03.601302+0000","last_clean":"2023-10-06T11:23:03.601302+0000","last_became_active":"2023-10-06T11:23:03.599478+0000","last_became_peered":"2023-10-06T11:23:03.599478+0000","last_unstale":"2023-10-06T11:23:03.601302+0000","last_undegraded":"2023-10-06T11:23:03.601302+0000","last_fullsized":"2023-10-06T11:23:03.601302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:00.230481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.577772+0000","last_change":"2023-10-06T11:23:02.569019+0000","last_active":"2023-10-06T11:23:03.577772+0000","last_peered":"2023-10-06T11:23:03.577772+0000","last_clean":"2023-10-06T11:23:03.577772+0000","last_became_active":"2023-10-06T11:23:02.568670+0000","last_became_peered":"2023-10-06T11:23:02.568670+0000","last_unstale":"2023-10-06T11:23:03.577772+0000","last_undegraded":"2023-10-06T11:23:03.577772+0000","last_fullsized":"2023-10-06T11:23:03.577772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:41.553006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581029+0000","last_change":"2023-10-06T11:23:03.581029+0000","last_active":"2023-10-06T11:23:03.581029+0000","last_peered":"2023-10-06T11:23:03.581029+0000","last_clean":"2023-10-06T11:23:03.581029+0000","last_became_active":"2023-10-06T11:23:03.576455+0000","last_became_peered":"2023-10-06T11:23:03.576455+0000","last_unstale":"2023-10-06T11:23:03.581029+0000","last_undegraded":"2023-10-06T11:23:03.581029+0000","last_fullsized":"2023-10-06T11:23:03.581029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:07.586839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.584336+0000","last_change":"2023-10-06T11:23:02.580722+0000","last_active":"2023-10-06T11:23:02.584336+0000","last_peered":"2023-10-06T11:23:02.584336+0000","last_clean":"2023-10-06T11:23:02.584336+0000","last_became_active":"2023-10-06T11:23:02.580440+0000","last_became_peered":"2023-10-06T11:23:02.580440+0000","last_unstale":"2023-10-06T11:23:02.584336+0000","last_undegraded":"2023-10-06T11:23:02.584336+0000","last_fullsized":"2023-10-06T11:23:02.584336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:30.598999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.575167+0000","last_change":"2023-10-06T11:23:03.572491+0000","last_active":"2023-10-06T11:23:03.575167+0000","last_peered":"2023-10-06T11:23:03.575167+0000","last_clean":"2023-10-06T11:23:03.575167+0000","last_became_active":"2023-10-06T11:23:03.572133+0000","last_became_peered":"2023-10-06T11:23:03.572133+0000","last_unstale":"2023-10-06T11:23:03.575167+0000","last_undegraded":"2023-10-06T11:23:03.575167+0000","last_fullsized":"2023-10-06T11:23:03.575167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600068+0000","last_change":"2023-10-06T11:23:02.586384+0000","last_active":"2023-10-06T11:23:03.600068+0000","last_peered":"2023-10-06T11:23:03.600068+0000","last_clean":"2023-10-06T11:23:03.600068+0000","last_became_active":"2023-10-06T11:23:02.585802+0000","last_became_peered":"2023-10-06T11:23:02.585802+0000","last_unstale":"2023-10-06T11:23:03.600068+0000","last_undegraded":"2023-10-06T11:23:03.600068+0000","last_fullsized":"2023-10-06T11:23:03.600068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.191497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584094+0000","last_change":"2023-10-06T11:23:03.584094+0000","last_active":"2023-10-06T11:23:03.584094+0000","last_peered":"2023-10-06T11:23:03.584094+0000","last_clean":"2023-10-06T11:23:03.584094+0000","last_became_active":"2023-10-06T11:23:03.583806+0000","last_became_peered":"2023-10-06T11:23:03.583806+0000","last_unstale":"2023-10-06T11:23:03.584094+0000","last_undegraded":"2023-10-06T11:23:03.584094+0000","last_fullsized":"2023-10-06T11:23:03.584094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604717+0000","last_change":"2023-10-06T11:23:02.571591+0000","last_active":"2023-10-06T11:23:03.604717+0000","last_peered":"2023-10-06T11:23:03.604717+0000","last_clean":"2023-10-06T11:23:03.604717+0000","last_became_active":"2023-10-06T11:23:02.570952+0000","last_became_peered":"2023-10-06T11:23:02.570952+0000","last_unstale":"2023-10-06T11:23:03.604717+0000","last_undegraded":"2023-10-06T11:23:03.604717+0000","last_fullsized":"2023-10-06T11:23:03.604717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:47.697735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593969+0000","last_change":"2023-10-06T11:23:03.591947+0000","last_active":"2023-10-06T11:23:03.593969+0000","last_peered":"2023-10-06T11:23:03.593969+0000","last_clean":"2023-10-06T11:23:03.593969+0000","last_became_active":"2023-10-06T11:23:03.591616+0000","last_became_peered":"2023-10-06T11:23:03.591616+0000","last_unstale":"2023-10-06T11:23:03.593969+0000","last_undegraded":"2023-10-06T11:23:03.593969+0000","last_fullsized":"2023-10-06T11:23:03.593969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:31.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604950+0000","last_change":"2023-10-06T11:23:02.582906+0000","last_active":"2023-10-06T11:23:03.604950+0000","last_peered":"2023-10-06T11:23:03.604950+0000","last_clean":"2023-10-06T11:23:03.604950+0000","last_became_active":"2023-10-06T11:23:02.582575+0000","last_became_peered":"2023-10-06T11:23:02.582575+0000","last_unstale":"2023-10-06T11:23:03.604950+0000","last_undegraded":"2023-10-06T11:23:03.604950+0000","last_fullsized":"2023-10-06T11:23:03.604950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.191497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589547+0000","last_change":"2023-10-06T11:23:03.589547+0000","last_active":"2023-10-06T11:23:03.589547+0000","last_peered":"2023-10-06T11:23:03.589547+0000","last_clean":"2023-10-06T11:23:03.589547+0000","last_became_active":"2023-10-06T11:23:03.586698+0000","last_became_peered":"2023-10-06T11:23:03.586698+0000","last_unstale":"2023-10-06T11:23:03.589547+0000","last_undegraded":"2023-10-06T11:23:03.589547+0000","last_fullsized":"2023-10-06T11:23:03.589547+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586980+0000","last_change":"2023-10-06T11:23:02.580420+0000","last_active":"2023-10-06T11:23:02.586980+0000","last_peered":"2023-10-06T11:23:02.586980+0000","last_clean":"2023-10-06T11:23:02.586980+0000","last_became_active":"2023-10-06T11:23:02.580129+0000","last_became_peered":"2023-10-06T11:23:02.580129+0000","last_unstale":"2023-10-06T11:23:02.586980+0000","last_undegraded":"2023-10-06T11:23:02.586980+0000","last_fullsized":"2023-10-06T11:23:02.586980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.256819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584821+0000","last_change":"2023-10-06T11:23:03.580096+0000","last_active":"2023-10-06T11:23:03.584821+0000","last_peered":"2023-10-06T11:23:03.584821+0000","last_clean":"2023-10-06T11:23:03.584821+0000","last_became_active":"2023-10-06T11:23:03.579809+0000","last_became_peered":"2023-10-06T11:23:03.579809+0000","last_unstale":"2023-10-06T11:23:03.584821+0000","last_undegraded":"2023-10-06T11:23:03.584821+0000","last_fullsized":"2023-10-06T11:23:03.584821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:02.622701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.564983+0000","last_change":"2023-10-06T11:23:02.578903+0000","last_active":"2023-10-06T11:23:03.564983+0000","last_peered":"2023-10-06T11:23:03.564983+0000","last_clean":"2023-10-06T11:23:03.564983+0000","last_became_active":"2023-10-06T11:23:02.578571+0000","last_became_peered":"2023-10-06T11:23:02.578571+0000","last_unstale":"2023-10-06T11:23:03.564983+0000","last_undegraded":"2023-10-06T11:23:03.564983+0000","last_fullsized":"2023-10-06T11:23:03.564983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:05:23.914766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581721+0000","last_change":"2023-10-06T11:23:03.581721+0000","last_active":"2023-10-06T11:23:03.581721+0000","last_peered":"2023-10-06T11:23:03.581721+0000","last_clean":"2023-10-06T11:23:03.581721+0000","last_became_active":"2023-10-06T11:23:03.581501+0000","last_became_peered":"2023-10-06T11:23:03.581501+0000","last_unstale":"2023-10-06T11:23:03.581721+0000","last_undegraded":"2023-10-06T11:23:03.581721+0000","last_fullsized":"2023-10-06T11:23:03.581721+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:18.747941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.607932+0000","last_change":"2023-10-06T11:23:02.579294+0000","last_active":"2023-10-06T11:23:03.607932+0000","last_peered":"2023-10-06T11:23:03.607932+0000","last_clean":"2023-10-06T11:23:03.607932+0000","last_became_active":"2023-10-06T11:23:02.578991+0000","last_became_peered":"2023-10-06T11:23:02.578991+0000","last_unstale":"2023-10-06T11:23:03.607932+0000","last_undegraded":"2023-10-06T11:23:03.607932+0000","last_fullsized":"2023-10-06T11:23:03.607932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:56.297774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.578913+0000","last_change":"2023-10-06T11:23:03.576996+0000","last_active":"2023-10-06T11:23:03.578913+0000","last_peered":"2023-10-06T11:23:03.578913+0000","last_clean":"2023-10-06T11:23:03.578913+0000","last_became_active":"2023-10-06T11:23:03.576437+0000","last_became_peered":"2023-10-06T11:23:03.576437+0000","last_unstale":"2023-10-06T11:23:03.578913+0000","last_undegraded":"2023-10-06T11:23:03.578913+0000","last_fullsized":"2023-10-06T11:23:03.578913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:36.098524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.602299+0000","last_change":"2023-10-06T11:23:02.573627+0000","last_active":"2023-10-06T11:23:03.602299+0000","last_peered":"2023-10-06T11:23:03.602299+0000","last_clean":"2023-10-06T11:23:03.602299+0000","last_became_active":"2023-10-06T11:23:02.573286+0000","last_became_peered":"2023-10-06T11:23:02.573286+0000","last_unstale":"2023-10-06T11:23:03.602299+0000","last_undegraded":"2023-10-06T11:23:03.602299+0000","last_fullsized":"2023-10-06T11:23:03.602299+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:51.056271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591711+0000","last_change":"2023-10-06T11:23:03.591711+0000","last_active":"2023-10-06T11:23:03.591711+0000","last_peered":"2023-10-06T11:23:03.591711+0000","last_clean":"2023-10-06T11:23:03.591711+0000","last_became_active":"2023-10-06T11:23:03.591353+0000","last_became_peered":"2023-10-06T11:23:03.591353+0000","last_unstale":"2023-10-06T11:23:03.591711+0000","last_undegraded":"2023-10-06T11:23:03.591711+0000","last_fullsized":"2023-10-06T11:23:03.591711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:44.048834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.605132+0000","last_change":"2023-10-06T11:23:02.575738+0000","last_active":"2023-10-06T11:23:03.605132+0000","last_peered":"2023-10-06T11:23:03.605132+0000","last_clean":"2023-10-06T11:23:03.605132+0000","last_became_active":"2023-10-06T11:23:02.575192+0000","last_became_peered":"2023-10-06T11:23:02.575192+0000","last_unstale":"2023-10-06T11:23:03.605132+0000","last_undegraded":"2023-10-06T11:23:03.605132+0000","last_fullsized":"2023-10-06T11:23:03.605132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:43.037158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.572180+0000","last_change":"2023-10-06T11:23:03.570835+0000","last_active":"2023-10-06T11:23:03.572180+0000","last_peered":"2023-10-06T11:23:03.572180+0000","last_clean":"2023-10-06T11:23:03.572180+0000","last_became_active":"2023-10-06T11:23:03.570489+0000","last_became_peered":"2023-10-06T11:23:03.570489+0000","last_unstale":"2023-10-06T11:23:03.572180+0000","last_undegraded":"2023-10-06T11:23:03.572180+0000","last_fullsized":"2023-10-06T11:23:03.572180+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:02.611041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574862+0000","last_change":"2023-10-06T11:23:02.582228+0000","last_active":"2023-10-06T11:23:03.574862+0000","last_peered":"2023-10-06T11:23:03.574862+0000","last_clean":"2023-10-06T11:23:03.574862+0000","last_became_active":"2023-10-06T11:23:02.581935+0000","last_became_peered":"2023-10-06T11:23:02.581935+0000","last_unstale":"2023-10-06T11:23:03.574862+0000","last_undegraded":"2023-10-06T11:23:03.574862+0000","last_fullsized":"2023-10-06T11:23:03.574862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:30.598999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593406+0000","last_change":"2023-10-06T11:23:03.583310+0000","last_active":"2023-10-06T11:23:03.593406+0000","last_peered":"2023-10-06T11:23:03.593406+0000","last_clean":"2023-10-06T11:23:03.593406+0000","last_became_active":"2023-10-06T11:23:03.582899+0000","last_became_peered":"2023-10-06T11:23:03.582899+0000","last_unstale":"2023-10-06T11:23:03.593406+0000","last_undegraded":"2023-10-06T11:23:03.593406+0000","last_fullsized":"2023-10-06T11:23:03.593406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:43.200957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.563917+0000","last_change":"2023-10-06T11:23:02.577006+0000","last_active":"2023-10-06T11:23:03.563917+0000","last_peered":"2023-10-06T11:23:03.563917+0000","last_clean":"2023-10-06T11:23:03.563917+0000","last_became_active":"2023-10-06T11:23:02.576588+0000","last_became_peered":"2023-10-06T11:23:02.576588+0000","last_unstale":"2023-10-06T11:23:03.563917+0000","last_undegraded":"2023-10-06T11:23:03.563917+0000","last_fullsized":"2023-10-06T11:23:03.563917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:23.201591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.575338+0000","last_change":"2023-10-06T11:23:03.572370+0000","last_active":"2023-10-06T11:23:03.575338+0000","last_peered":"2023-10-06T11:23:03.575338+0000","last_clean":"2023-10-06T11:23:03.575338+0000","last_became_active":"2023-10-06T11:23:03.572053+0000","last_became_peered":"2023-10-06T11:23:03.572053+0000","last_unstale":"2023-10-06T11:23:03.575338+0000","last_undegraded":"2023-10-06T11:23:03.575338+0000","last_fullsized":"2023-10-06T11:23:03.575338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600291+0000","last_change":"2023-10-06T11:23:02.573151+0000","last_active":"2023-10-06T11:23:03.600291+0000","last_peered":"2023-10-06T11:23:03.600291+0000","last_clean":"2023-10-06T11:23:03.600291+0000","last_became_active":"2023-10-06T11:23:02.572186+0000","last_became_peered":"2023-10-06T11:23:02.572186+0000","last_unstale":"2023-10-06T11:23:03.600291+0000","last_undegraded":"2023-10-06T11:23:03.600291+0000","last_fullsized":"2023-10-06T11:23:03.600291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.262517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.594176+0000","last_change":"2023-10-06T11:23:03.594176+0000","last_active":"2023-10-06T11:23:03.594176+0000","last_peered":"2023-10-06T11:23:03.594176+0000","last_clean":"2023-10-06T11:23:03.594176+0000","last_became_active":"2023-10-06T11:23:03.593888+0000","last_became_peered":"2023-10-06T11:23:03.593888+0000","last_unstale":"2023-10-06T11:23:03.594176+0000","last_undegraded":"2023-10-06T11:23:03.594176+0000","last_fullsized":"2023-10-06T11:23:03.594176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:43.080095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.583902+0000","last_change":"2023-10-06T11:23:02.581847+0000","last_active":"2023-10-06T11:23:02.583902+0000","last_peered":"2023-10-06T11:23:02.583902+0000","last_clean":"2023-10-06T11:23:02.583902+0000","last_became_active":"2023-10-06T11:23:02.581488+0000","last_became_peered":"2023-10-06T11:23:02.581488+0000","last_unstale":"2023-10-06T11:23:02.583902+0000","last_undegraded":"2023-10-06T11:23:02.583902+0000","last_fullsized":"2023-10-06T11:23:02.583902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.579355+0000","last_change":"2023-10-06T11:23:03.578701+0000","last_active":"2023-10-06T11:23:03.579355+0000","last_peered":"2023-10-06T11:23:03.579355+0000","last_clean":"2023-10-06T11:23:03.579355+0000","last_became_active":"2023-10-06T11:23:03.578413+0000","last_became_peered":"2023-10-06T11:23:03.578413+0000","last_unstale":"2023-10-06T11:23:03.579355+0000","last_undegraded":"2023-10-06T11:23:03.579355+0000","last_fullsized":"2023-10-06T11:23:03.579355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:24.416766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.584483+0000","last_change":"2023-10-06T11:23:02.581460+0000","last_active":"2023-10-06T11:23:02.584483+0000","last_peered":"2023-10-06T11:23:02.584483+0000","last_clean":"2023-10-06T11:23:02.584483+0000","last_became_active":"2023-10-06T11:23:02.580546+0000","last_became_peered":"2023-10-06T11:23:02.580546+0000","last_unstale":"2023-10-06T11:23:02.584483+0000","last_undegraded":"2023-10-06T11:23:02.584483+0000","last_fullsized":"2023-10-06T11:23:02.584483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:45.908974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.595176+0000","last_change":"2023-10-06T11:23:03.595176+0000","last_active":"2023-10-06T11:23:03.595176+0000","last_peered":"2023-10-06T11:23:03.595176+0000","last_clean":"2023-10-06T11:23:03.595176+0000","last_became_active":"2023-10-06T11:23:03.593091+0000","last_became_peered":"2023-10-06T11:23:03.593091+0000","last_unstale":"2023-10-06T11:23:03.595176+0000","last_undegraded":"2023-10-06T11:23:03.595176+0000","last_fullsized":"2023-10-06T11:23:03.595176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:54.497474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585676+0000","last_change":"2023-10-06T11:23:02.578781+0000","last_active":"2023-10-06T11:23:02.585676+0000","last_peered":"2023-10-06T11:23:02.585676+0000","last_clean":"2023-10-06T11:23:02.585676+0000","last_became_active":"2023-10-06T11:23:02.578475+0000","last_became_peered":"2023-10-06T11:23:02.578475+0000","last_unstale":"2023-10-06T11:23:02.585676+0000","last_undegraded":"2023-10-06T11:23:02.585676+0000","last_fullsized":"2023-10-06T11:23:02.585676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:08.736830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585748+0000","last_change":"2023-10-06T11:23:03.583220+0000","last_active":"2023-10-06T11:23:03.585748+0000","last_peered":"2023-10-06T11:23:03.585748+0000","last_clean":"2023-10-06T11:23:03.585748+0000","last_became_active":"2023-10-06T11:23:03.582786+0000","last_became_peered":"2023-10-06T11:23:03.582786+0000","last_unstale":"2023-10-06T11:23:03.585748+0000","last_undegraded":"2023-10-06T11:23:03.585748+0000","last_fullsized":"2023-10-06T11:23:03.585748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:02.622701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585316+0000","last_change":"2023-10-06T11:23:02.583486+0000","last_active":"2023-10-06T11:23:02.585316+0000","last_peered":"2023-10-06T11:23:02.585316+0000","last_clean":"2023-10-06T11:23:02.585316+0000","last_became_active":"2023-10-06T11:23:02.583093+0000","last_became_peered":"2023-10-06T11:23:02.583093+0000","last_unstale":"2023-10-06T11:23:02.585316+0000","last_undegraded":"2023-10-06T11:23:02.585316+0000","last_fullsized":"2023-10-06T11:23:02.585316+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.567567+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:03.567567+0000","last_peered":"2023-10-06T11:23:03.567567+0000","last_clean":"2023-10-06T11:23:03.567567+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:03.567567+0000","last_undegraded":"2023-10-06T11:23:03.567567+0000","last_fullsized":"2023-10-06T11:23:03.567567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584732+0000","last_change":"2023-10-06T11:23:03.584732+0000","last_active":"2023-10-06T11:23:03.584732+0000","last_peered":"2023-10-06T11:23:03.584732+0000","last_clean":"2023-10-06T11:23:03.584732+0000","last_became_active":"2023-10-06T11:23:03.584369+0000","last_became_peered":"2023-10-06T11:23:03.584369+0000","last_unstale":"2023-10-06T11:23:03.584732+0000","last_undegraded":"2023-10-06T11:23:03.584732+0000","last_fullsized":"2023-10-06T11:23:03.584732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:59.274194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586447+0000","last_change":"2023-10-06T11:23:02.585146+0000","last_active":"2023-10-06T11:23:02.586447+0000","last_peered":"2023-10-06T11:23:02.586447+0000","last_clean":"2023-10-06T11:23:02.586447+0000","last_became_active":"2023-10-06T11:23:02.584815+0000","last_became_peered":"2023-10-06T11:23:02.584815+0000","last_unstale":"2023-10-06T11:23:02.586447+0000","last_undegraded":"2023-10-06T11:23:02.586447+0000","last_fullsized":"2023-10-06T11:23:02.586447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:42.189280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.566539+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:03.566539+0000","last_peered":"2023-10-06T11:23:03.566539+0000","last_clean":"2023-10-06T11:23:03.566539+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:03.566539+0000","last_undegraded":"2023-10-06T11:23:03.566539+0000","last_fullsized":"2023-10-06T11:23:03.566539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.586094+0000","last_change":"2023-10-06T11:23:03.580448+0000","last_active":"2023-10-06T11:23:03.586094+0000","last_peered":"2023-10-06T11:23:03.586094+0000","last_clean":"2023-10-06T11:23:03.586094+0000","last_became_active":"2023-10-06T11:23:03.580126+0000","last_became_peered":"2023-10-06T11:23:03.580126+0000","last_unstale":"2023-10-06T11:23:03.586094+0000","last_undegraded":"2023-10-06T11:23:03.586094+0000","last_fullsized":"2023-10-06T11:23:03.586094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:19.535711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.579947+0000","last_change":"2023-10-06T11:23:02.573383+0000","last_active":"2023-10-06T11:23:03.579947+0000","last_peered":"2023-10-06T11:23:03.579947+0000","last_clean":"2023-10-06T11:23:03.579947+0000","last_became_active":"2023-10-06T11:23:02.573018+0000","last_became_peered":"2023-10-06T11:23:02.573018+0000","last_unstale":"2023-10-06T11:23:03.579947+0000","last_undegraded":"2023-10-06T11:23:03.579947+0000","last_fullsized":"2023-10-06T11:23:03.579947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:30.773636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.572622+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:03.572622+0000","last_peered":"2023-10-06T11:23:03.572622+0000","last_clean":"2023-10-06T11:23:03.572622+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:03.572622+0000","last_undegraded":"2023-10-06T11:23:03.572622+0000","last_fullsized":"2023-10-06T11:23:03.572622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587681+0000","last_change":"2023-10-06T11:23:03.586717+0000","last_active":"2023-10-06T11:23:03.587681+0000","last_peered":"2023-10-06T11:23:03.587681+0000","last_clean":"2023-10-06T11:23:03.587681+0000","last_became_active":"2023-10-06T11:23:03.586428+0000","last_became_peered":"2023-10-06T11:23:03.586428+0000","last_unstale":"2023-10-06T11:23:03.587681+0000","last_undegraded":"2023-10-06T11:23:03.587681+0000","last_fullsized":"2023-10-06T11:23:03.587681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.587162+0000","last_change":"2023-10-06T11:23:02.583977+0000","last_active":"2023-10-06T11:23:02.587162+0000","last_peered":"2023-10-06T11:23:02.587162+0000","last_clean":"2023-10-06T11:23:02.587162+0000","last_became_active":"2023-10-06T11:23:02.583629+0000","last_became_peered":"2023-10-06T11:23:02.583629+0000","last_unstale":"2023-10-06T11:23:02.587162+0000","last_undegraded":"2023-10-06T11:23:02.587162+0000","last_fullsized":"2023-10-06T11:23:02.587162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:27.024345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.564127+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:03.564127+0000","last_peered":"2023-10-06T11:23:03.564127+0000","last_clean":"2023-10-06T11:23:03.564127+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:03.564127+0000","last_undegraded":"2023-10-06T11:23:03.564127+0000","last_fullsized":"2023-10-06T11:23:03.564127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.578331+0000","last_change":"2023-10-06T11:23:03.578331+0000","last_active":"2023-10-06T11:23:03.578331+0000","last_peered":"2023-10-06T11:23:03.578331+0000","last_clean":"2023-10-06T11:23:03.578331+0000","last_became_active":"2023-10-06T11:23:03.578001+0000","last_became_peered":"2023-10-06T11:23:03.578001+0000","last_unstale":"2023-10-06T11:23:03.578331+0000","last_undegraded":"2023-10-06T11:23:03.578331+0000","last_fullsized":"2023-10-06T11:23:03.578331+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:54.497474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.582774+0000","last_change":"2023-10-06T11:23:02.581981+0000","last_active":"2023-10-06T11:23:02.582774+0000","last_peered":"2023-10-06T11:23:02.582774+0000","last_clean":"2023-10-06T11:23:02.582774+0000","last_became_active":"2023-10-06T11:23:02.581679+0000","last_became_peered":"2023-10-06T11:23:02.581679+0000","last_unstale":"2023-10-06T11:23:02.582774+0000","last_undegraded":"2023-10-06T11:23:02.582774+0000","last_fullsized":"2023-10-06T11:23:02.582774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:26.500019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.570952+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:02.570952+0000","last_peered":"2023-10-06T11:23:02.570952+0000","last_clean":"2023-10-06T11:23:02.570952+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:02.570952+0000","last_undegraded":"2023-10-06T11:23:02.570952+0000","last_fullsized":"2023-10-06T11:23:02.570952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.573217+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:02.573217+0000","last_peered":"2023-10-06T11:23:02.573217+0000","last_clean":"2023-10-06T11:23:02.573217+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:02.573217+0000","last_undegraded":"2023-10-06T11:23:02.573217+0000","last_fullsized":"2023-10-06T11:23:02.573217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592053+0000","last_change":"2023-10-06T11:23:03.592053+0000","last_active":"2023-10-06T11:23:03.592053+0000","last_peered":"2023-10-06T11:23:03.592053+0000","last_clean":"2023-10-06T11:23:03.592053+0000","last_became_active":"2023-10-06T11:23:03.590737+0000","last_became_peered":"2023-10-06T11:23:03.590737+0000","last_unstale":"2023-10-06T11:23:03.592053+0000","last_undegraded":"2023-10-06T11:23:03.592053+0000","last_fullsized":"2023-10-06T11:23:03.592053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:34.030555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.587582+0000","last_change":"2023-10-06T11:23:02.586417+0000","last_active":"2023-10-06T11:23:02.587582+0000","last_peered":"2023-10-06T11:23:02.587582+0000","last_clean":"2023-10-06T11:23:02.587582+0000","last_became_active":"2023-10-06T11:23:02.586131+0000","last_became_peered":"2023-10-06T11:23:02.586131+0000","last_unstale":"2023-10-06T11:23:02.587582+0000","last_undegraded":"2023-10-06T11:23:02.587582+0000","last_fullsized":"2023-10-06T11:23:02.587582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.003668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.574627+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:02.574627+0000","last_peered":"2023-10-06T11:23:02.574627+0000","last_clean":"2023-10-06T11:23:02.574627+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:02.574627+0000","last_undegraded":"2023-10-06T11:23:02.574627+0000","last_fullsized":"2023-10-06T11:23:02.574627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.596975+0000","last_change":"2023-10-06T11:23:03.593178+0000","last_active":"2023-10-06T11:23:03.596975+0000","last_peered":"2023-10-06T11:23:03.596975+0000","last_clean":"2023-10-06T11:23:03.596975+0000","last_became_active":"2023-10-06T11:23:03.592800+0000","last_became_peered":"2023-10-06T11:23:03.592800+0000","last_unstale":"2023-10-06T11:23:03.596975+0000","last_undegraded":"2023-10-06T11:23:03.596975+0000","last_fullsized":"2023-10-06T11:23:03.596975+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:43.080095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.565130+0000","last_change":"2023-10-06T11:23:02.571930+0000","last_active":"2023-10-06T11:23:03.565130+0000","last_peered":"2023-10-06T11:23:03.565130+0000","last_clean":"2023-10-06T11:23:03.565130+0000","last_became_active":"2023-10-06T11:23:02.571593+0000","last_became_peered":"2023-10-06T11:23:02.571593+0000","last_unstale":"2023-10-06T11:23:03.565130+0000","last_undegraded":"2023-10-06T11:23:03.565130+0000","last_fullsized":"2023-10-06T11:23:03.565130+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.003668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.569581+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:02.569581+0000","last_peered":"2023-10-06T11:23:02.569581+0000","last_clean":"2023-10-06T11:23:02.569581+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:02.569581+0000","last_undegraded":"2023-10-06T11:23:02.569581+0000","last_fullsized":"2023-10-06T11:23:02.569581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599137+0000","last_change":"2023-10-06T11:23:03.599137+0000","last_active":"2023-10-06T11:23:03.599137+0000","last_peered":"2023-10-06T11:23:03.599137+0000","last_clean":"2023-10-06T11:23:03.599137+0000","last_became_active":"2023-10-06T11:23:03.598829+0000","last_became_peered":"2023-10-06T11:23:03.598829+0000","last_unstale":"2023-10-06T11:23:03.599137+0000","last_undegraded":"2023-10-06T11:23:03.599137+0000","last_fullsized":"2023-10-06T11:23:03.599137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600001+0000","last_change":"2023-10-06T11:23:02.582219+0000","last_active":"2023-10-06T11:23:03.600001+0000","last_peered":"2023-10-06T11:23:03.600001+0000","last_clean":"2023-10-06T11:23:03.600001+0000","last_became_active":"2023-10-06T11:23:02.581848+0000","last_became_peered":"2023-10-06T11:23:02.581848+0000","last_unstale":"2023-10-06T11:23:03.600001+0000","last_undegraded":"2023-10-06T11:23:03.600001+0000","last_fullsized":"2023-10-06T11:23:03.600001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:45.908974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.571471+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:02.571471+0000","last_peered":"2023-10-06T11:23:02.571471+0000","last_clean":"2023-10-06T11:23:02.571471+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:02.571471+0000","last_undegraded":"2023-10-06T11:23:02.571471+0000","last_fullsized":"2023-10-06T11:23:02.571471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1036288,"data_stored":785500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.98399999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0529999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.94899999999999995}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0820000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301820,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070020,"statfs":{"total":96636764160,"available":96327700480,"internally_reserved":0,"allocated":446464,"data_stored":195686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85299999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.0129999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.266},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.2250000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.6850000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.573},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.617},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.738}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":442368,"data_stored":195112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.746},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.02}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.079},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.87},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.21}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301836,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070004,"statfs":{"total":96636764160,"available":96327684096,"internally_reserved":0,"allocated":462848,"data_stored":203539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.256},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.93300000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.998},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":1.1699999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.286},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.125}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":1.385},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.351}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.329}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301836,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070004,"statfs":{"total":96636764160,"available":96327684096,"internally_reserved":0,"allocated":462848,"data_stored":202613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.874},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1679999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.044}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.117},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301840,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070000,"statfs":{"total":96636764160,"available":96327680000,"internally_reserved":0,"allocated":466944,"data_stored":203561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.0629999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1036288,"data_stored":785480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.998},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":442368,"data_stored":195076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.93200000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.163},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0900000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0649999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.137}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:07.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:07.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 Processor -- start 2023-10-06T11:23:07.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 -- start start 2023-10-06T11:23:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 0x7fb71c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb71c0ad990 0x7fb71c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71c0c80a0 con 0x7fb71c06d2f0 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71c15d470 con 0x7fb71c0ad990 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722a91700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c15d750 con 0x7fb71c06be20 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722290700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb71c0ad990 0x7fb71c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb721a8f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 0x7fb71c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722290700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb71c0ad990 0x7fb71c1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54192/0 (socket says 172.21.15.7:54192) 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb722290700 1 -- 172.21.15.7:0/4195927034 learned_addr learned my addr 172.21.15.7:0/4195927034 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.703+0000 7fb72128e700 1 --1- 172.21.15.7:0/4195927034 >> v1:172.21.15.50:6789/0 conn(0x7fb71c06d2f0 0x7fb71c15cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:56154/0 (socket says 172.21.15.7:56154) 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb720a8d700 1 -- 172.21.15.7:0/4195927034 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527460935 0 0) 0x7fb71c0c80a0 con 0x7fb71c06d2f0 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb720a8d700 1 -- 172.21.15.7:0/4195927034 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb708003570 con 0x7fb71c06d2f0 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722290700 1 -- 172.21.15.7:0/4195927034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 msgr2=0x7fb71c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722290700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 0x7fb71c06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722290700 1 -- 172.21.15.7:0/4195927034 >> v1:172.21.15.50:6789/0 conn(0x7fb71c06d2f0 legacy=0x7fb71c15cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722290700 1 -- 172.21.15.7:0/4195927034 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c15da30 con 0x7fb71c0ad990 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb721a8f700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 0x7fb71c06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722290700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1589a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb7140099b0 tx=0x7fb714009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062442da54ba22b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb720a8d700 1 -- 172.21.15.7:0/4195927034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400b680 con 0x7fb71c0ad990 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb720a8d700 1 -- 172.21.15.7:0/4195927034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400b7e0 con 0x7fb71c0ad990 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb720a8d700 1 -- 172.21.15.7:0/4195927034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400bba0 con 0x7fb71c0ad990 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- 172.21.15.7:0/4195927034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 msgr2=0x7fb71c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1589a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb7140099b0 tx=0x7fb714009250 comp rx=0 tx=0).stop 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- 172.21.15.7:0/4195927034 shutdown_connections 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1589a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 --2- 172.21.15.7:0/4195927034 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06be20 0x7fb71c06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- 172.21.15.7:0/4195927034 >> 172.21.15.7:0/4195927034 conn(0x7fb71c0cdb00 msgr2=0x7fb71c0cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- 172.21.15.7:0/4195927034 shutdown_connections 2023-10-06T11:23:07.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- 172.21.15.7:0/4195927034 wait complete. 2023-10-06T11:23:07.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 Processor -- start 2023-10-06T11:23:07.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.707+0000 7fb722a91700 1 -- start start 2023-10-06T11:23:07.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb71c06be20 0x7fb71c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 0x7fb71c15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c156c50 con 0x7fb71c0ad990 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0b0c80 con 0x7fb71c06be20 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0b0950 con 0x7fb71c06d2f0 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54200/0 (socket says 172.21.15.7:54200) 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 -- 172.21.15.7:0/2782512766 learned_addr learned my addr 172.21.15.7:0/2782512766 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb72128e700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 0x7fb71c15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb721a8f700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb71c06be20 0x7fb71c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 -- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 msgr2=0x7fb71c15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 0x7fb71c15b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 -- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb71c06be20 msgr2=0x7fb71c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb71c06be20 0x7fb71c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 -- 172.21.15.7:0/2782512766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c009420 con 0x7fb71c0ad990 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb72128e700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 0x7fb71c15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:07.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722290700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb714006fd0 tx=0x7fb714017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400bb60 con 0x7fb71c0ad990 2023-10-06T11:23:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71401a430 con 0x7fb71c0ad990 2023-10-06T11:23:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7140094f0 con 0x7fb71c0ad990 2023-10-06T11:23:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71401a7d0 con 0x7fb71c0ad990 2023-10-06T11:23:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.711+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c15c0a0 con 0x7fb71c0ad990 2023-10-06T11:23:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c06e8e0 con 0x7fb71c0ad990 2023-10-06T11:23:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb71401ab80 con 0x7fb71c0ad990 2023-10-06T11:23:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb711ffb700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 0x7fb700062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb71407e730 con 0x7fb71c0ad990 2023-10-06T11:23:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb721a8f700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 0x7fb700062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:07.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.715+0000 7fb721a8f700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 0x7fb700062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0093f0 tx=0x7fb70c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.723+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb71404ea80 con 0x7fb71c0ad990 2023-10-06T11:23:07.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.883+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb71c0c82f0 con 0x7fb700060550 2023-10-06T11:23:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.899+0000 7fb711ffb700 1 -- 172.21.15.7:0/2782512766 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309525 (secure 0 0 0) 0x7fb71c0c82f0 con 0x7fb700060550 2023-10-06T11:23:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 msgr2=0x7fb700062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 0x7fb700062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0093f0 tx=0x7fb70c008040 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 msgr2=0x7fb71c1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb714006fd0 tx=0x7fb714017000 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 shutdown_connections 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb700060550 0x7fb700062a00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c0ad990 0x7fb71c1571f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb71c06d2f0 0x7fb71c15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 --2- 172.21.15.7:0/2782512766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb71c06be20 0x7fb71c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 >> 172.21.15.7:0/2782512766 conn(0x7fb71c0cdb00 msgr2=0x7fb71c0ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 shutdown_connections 2023-10-06T11:23:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:07.903+0000 7fb722a91700 1 -- 172.21.15.7:0/2782512766 wait complete. 2023-10-06T11:23:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:07.921 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T11:23:07.837603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415756,"kb_used_data":4684,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752558964,"statfs":{"total":773094113280,"available":770620379136,"internally_reserved":0,"allocated":4796416,"data_stored":2766567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002195"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587558+0000","last_change":"2023-10-06T11:23:03.587558+0000","last_active":"2023-10-06T11:23:03.587558+0000","last_peered":"2023-10-06T11:23:03.587558+0000","last_clean":"2023-10-06T11:23:03.587558+0000","last_became_active":"2023-10-06T11:23:03.584983+0000","last_became_peered":"2023-10-06T11:23:03.584983+0000","last_unstale":"2023-10-06T11:23:03.587558+0000","last_undegraded":"2023-10-06T11:23:03.587558+0000","last_fullsized":"2023-10-06T11:23:03.587558+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593143+0000","last_change":"2023-10-06T11:23:03.593143+0000","last_active":"2023-10-06T11:23:03.593143+0000","last_peered":"2023-10-06T11:23:03.593143+0000","last_clean":"2023-10-06T11:23:03.593143+0000","last_became_active":"2023-10-06T11:23:03.592544+0000","last_became_peered":"2023-10-06T11:23:03.592544+0000","last_unstale":"2023-10-06T11:23:03.593143+0000","last_undegraded":"2023-10-06T11:23:03.593143+0000","last_fullsized":"2023-10-06T11:23:03.593143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:05:24.926443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585950+0000","last_change":"2023-10-06T11:23:03.582956+0000","last_active":"2023-10-06T11:23:03.585950+0000","last_peered":"2023-10-06T11:23:03.585950+0000","last_clean":"2023-10-06T11:23:03.585950+0000","last_became_active":"2023-10-06T11:23:03.582628+0000","last_became_peered":"2023-10-06T11:23:03.582628+0000","last_unstale":"2023-10-06T11:23:03.585950+0000","last_undegraded":"2023-10-06T11:23:03.585950+0000","last_fullsized":"2023-10-06T11:23:03.585950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591639+0000","last_change":"2023-10-06T11:23:03.591639+0000","last_active":"2023-10-06T11:23:03.591639+0000","last_peered":"2023-10-06T11:23:03.591639+0000","last_clean":"2023-10-06T11:23:03.591639+0000","last_became_active":"2023-10-06T11:23:03.591149+0000","last_became_peered":"2023-10-06T11:23:03.591149+0000","last_unstale":"2023-10-06T11:23:03.591639+0000","last_undegraded":"2023-10-06T11:23:03.591639+0000","last_fullsized":"2023-10-06T11:23:03.591639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:10.120082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592151+0000","last_change":"2023-10-06T11:23:03.592151+0000","last_active":"2023-10-06T11:23:03.592151+0000","last_peered":"2023-10-06T11:23:03.592151+0000","last_clean":"2023-10-06T11:23:03.592151+0000","last_became_active":"2023-10-06T11:23:03.591779+0000","last_became_peered":"2023-10-06T11:23:03.591779+0000","last_unstale":"2023-10-06T11:23:03.592151+0000","last_undegraded":"2023-10-06T11:23:03.592151+0000","last_fullsized":"2023-10-06T11:23:03.592151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.818097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581242+0000","last_change":"2023-10-06T11:23:03.578183+0000","last_active":"2023-10-06T11:23:03.581242+0000","last_peered":"2023-10-06T11:23:03.581242+0000","last_clean":"2023-10-06T11:23:03.581242+0000","last_became_active":"2023-10-06T11:23:03.577805+0000","last_became_peered":"2023-10-06T11:23:03.577805+0000","last_unstale":"2023-10-06T11:23:03.581242+0000","last_undegraded":"2023-10-06T11:23:03.581242+0000","last_fullsized":"2023-10-06T11:23:03.581242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:28.588171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584028+0000","last_change":"2023-10-06T11:23:03.584028+0000","last_active":"2023-10-06T11:23:03.584028+0000","last_peered":"2023-10-06T11:23:03.584028+0000","last_clean":"2023-10-06T11:23:03.584028+0000","last_became_active":"2023-10-06T11:23:03.583722+0000","last_became_peered":"2023-10-06T11:23:03.583722+0000","last_unstale":"2023-10-06T11:23:03.584028+0000","last_undegraded":"2023-10-06T11:23:03.584028+0000","last_fullsized":"2023-10-06T11:23:03.584028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589584+0000","last_change":"2023-10-06T11:23:03.589584+0000","last_active":"2023-10-06T11:23:03.589584+0000","last_peered":"2023-10-06T11:23:03.589584+0000","last_clean":"2023-10-06T11:23:03.589584+0000","last_became_active":"2023-10-06T11:23:03.588153+0000","last_became_peered":"2023-10-06T11:23:03.588153+0000","last_unstale":"2023-10-06T11:23:03.589584+0000","last_undegraded":"2023-10-06T11:23:03.589584+0000","last_fullsized":"2023-10-06T11:23:03.589584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598463+0000","last_change":"2023-10-06T11:23:03.598463+0000","last_active":"2023-10-06T11:23:03.598463+0000","last_peered":"2023-10-06T11:23:03.598463+0000","last_clean":"2023-10-06T11:23:03.598463+0000","last_became_active":"2023-10-06T11:23:03.598120+0000","last_became_peered":"2023-10-06T11:23:03.598120+0000","last_unstale":"2023-10-06T11:23:03.598463+0000","last_undegraded":"2023-10-06T11:23:03.598463+0000","last_fullsized":"2023-10-06T11:23:03.598463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:18.747941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599961+0000","last_change":"2023-10-06T11:23:03.599961+0000","last_active":"2023-10-06T11:23:03.599961+0000","last_peered":"2023-10-06T11:23:03.599961+0000","last_clean":"2023-10-06T11:23:03.599961+0000","last_became_active":"2023-10-06T11:23:03.599618+0000","last_became_peered":"2023-10-06T11:23:03.599618+0000","last_unstale":"2023-10-06T11:23:03.599961+0000","last_undegraded":"2023-10-06T11:23:03.599961+0000","last_fullsized":"2023-10-06T11:23:03.599961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:19.535711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598084+0000","last_change":"2023-10-06T11:23:03.598084+0000","last_active":"2023-10-06T11:23:03.598084+0000","last_peered":"2023-10-06T11:23:03.598084+0000","last_clean":"2023-10-06T11:23:03.598084+0000","last_became_active":"2023-10-06T11:23:03.597694+0000","last_became_peered":"2023-10-06T11:23:03.597694+0000","last_unstale":"2023-10-06T11:23:03.598084+0000","last_undegraded":"2023-10-06T11:23:03.598084+0000","last_fullsized":"2023-10-06T11:23:03.598084+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:35.666497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585891+0000","last_change":"2023-10-06T11:23:03.585891+0000","last_active":"2023-10-06T11:23:03.585891+0000","last_peered":"2023-10-06T11:23:03.585891+0000","last_clean":"2023-10-06T11:23:03.585891+0000","last_became_active":"2023-10-06T11:23:03.585297+0000","last_became_peered":"2023-10-06T11:23:03.585297+0000","last_unstale":"2023-10-06T11:23:03.585891+0000","last_undegraded":"2023-10-06T11:23:03.585891+0000","last_fullsized":"2023-10-06T11:23:03.585891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:14.203173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581685+0000","last_change":"2023-10-06T11:23:03.580379+0000","last_active":"2023-10-06T11:23:03.581685+0000","last_peered":"2023-10-06T11:23:03.581685+0000","last_clean":"2023-10-06T11:23:03.581685+0000","last_became_active":"2023-10-06T11:23:03.580083+0000","last_became_peered":"2023-10-06T11:23:03.580083+0000","last_unstale":"2023-10-06T11:23:03.581685+0000","last_undegraded":"2023-10-06T11:23:03.581685+0000","last_fullsized":"2023-10-06T11:23:03.581685+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:28:14.475031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.597730+0000","last_change":"2023-10-06T11:23:03.597730+0000","last_active":"2023-10-06T11:23:03.597730+0000","last_peered":"2023-10-06T11:23:03.597730+0000","last_clean":"2023-10-06T11:23:03.597730+0000","last_became_active":"2023-10-06T11:23:03.597423+0000","last_became_peered":"2023-10-06T11:23:03.597423+0000","last_unstale":"2023-10-06T11:23:03.597730+0000","last_undegraded":"2023-10-06T11:23:03.597730+0000","last_fullsized":"2023-10-06T11:23:03.597730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585328+0000","last_change":"2023-10-06T11:23:03.583822+0000","last_active":"2023-10-06T11:23:03.585328+0000","last_peered":"2023-10-06T11:23:03.585328+0000","last_clean":"2023-10-06T11:23:03.585328+0000","last_became_active":"2023-10-06T11:23:03.583571+0000","last_became_peered":"2023-10-06T11:23:03.583571+0000","last_unstale":"2023-10-06T11:23:03.585328+0000","last_undegraded":"2023-10-06T11:23:03.585328+0000","last_fullsized":"2023-10-06T11:23:03.585328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:33.820961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593956+0000","last_change":"2023-10-06T11:23:03.591839+0000","last_active":"2023-10-06T11:23:03.593956+0000","last_peered":"2023-10-06T11:23:03.593956+0000","last_clean":"2023-10-06T11:23:03.593956+0000","last_became_active":"2023-10-06T11:23:03.591528+0000","last_became_peered":"2023-10-06T11:23:03.591528+0000","last_unstale":"2023-10-06T11:23:03.593956+0000","last_undegraded":"2023-10-06T11:23:03.593956+0000","last_fullsized":"2023-10-06T11:23:03.593956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:41.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585798+0000","last_change":"2023-10-06T11:23:03.582937+0000","last_active":"2023-10-06T11:23:03.585798+0000","last_peered":"2023-10-06T11:23:03.585798+0000","last_clean":"2023-10-06T11:23:03.585798+0000","last_became_active":"2023-10-06T11:23:03.582598+0000","last_became_peered":"2023-10-06T11:23:03.582598+0000","last_unstale":"2023-10-06T11:23:03.585798+0000","last_undegraded":"2023-10-06T11:23:03.585798+0000","last_fullsized":"2023-10-06T11:23:03.585798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:44.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.594417+0000","last_change":"2023-10-06T11:23:03.592964+0000","last_active":"2023-10-06T11:23:03.594417+0000","last_peered":"2023-10-06T11:23:03.594417+0000","last_clean":"2023-10-06T11:23:03.594417+0000","last_became_active":"2023-10-06T11:23:03.592611+0000","last_became_peered":"2023-10-06T11:23:03.592611+0000","last_unstale":"2023-10-06T11:23:03.594417+0000","last_undegraded":"2023-10-06T11:23:03.594417+0000","last_fullsized":"2023-10-06T11:23:03.594417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:41.947324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.601395+0000","last_change":"2023-10-06T11:23:03.599987+0000","last_active":"2023-10-06T11:23:03.601395+0000","last_peered":"2023-10-06T11:23:03.601395+0000","last_clean":"2023-10-06T11:23:03.601395+0000","last_became_active":"2023-10-06T11:23:03.599658+0000","last_became_peered":"2023-10-06T11:23:03.599658+0000","last_unstale":"2023-10-06T11:23:03.601395+0000","last_undegraded":"2023-10-06T11:23:03.601395+0000","last_fullsized":"2023-10-06T11:23:03.601395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:57.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589149+0000","last_change":"2023-10-06T11:23:03.587617+0000","last_active":"2023-10-06T11:23:03.589149+0000","last_peered":"2023-10-06T11:23:03.589149+0000","last_clean":"2023-10-06T11:23:03.589149+0000","last_became_active":"2023-10-06T11:23:03.587337+0000","last_became_peered":"2023-10-06T11:23:03.587337+0000","last_unstale":"2023-10-06T11:23:03.589149+0000","last_undegraded":"2023-10-06T11:23:03.589149+0000","last_fullsized":"2023-10-06T11:23:03.589149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:02.611041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574406+0000","last_change":"2023-10-06T11:23:03.570300+0000","last_active":"2023-10-06T11:23:03.574406+0000","last_peered":"2023-10-06T11:23:03.574406+0000","last_clean":"2023-10-06T11:23:03.574406+0000","last_became_active":"2023-10-06T11:23:03.569859+0000","last_became_peered":"2023-10-06T11:23:03.569859+0000","last_unstale":"2023-10-06T11:23:03.574406+0000","last_undegraded":"2023-10-06T11:23:03.574406+0000","last_fullsized":"2023-10-06T11:23:03.574406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:33.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587337+0000","last_change":"2023-10-06T11:23:03.587337+0000","last_active":"2023-10-06T11:23:03.587337+0000","last_peered":"2023-10-06T11:23:03.587337+0000","last_clean":"2023-10-06T11:23:03.587337+0000","last_became_active":"2023-10-06T11:23:03.586816+0000","last_became_peered":"2023-10-06T11:23:03.586816+0000","last_unstale":"2023-10-06T11:23:03.587337+0000","last_undegraded":"2023-10-06T11:23:03.587337+0000","last_fullsized":"2023-10-06T11:23:03.587337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:07.586839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581935+0000","last_change":"2023-10-06T11:23:03.579258+0000","last_active":"2023-10-06T11:23:03.581935+0000","last_peered":"2023-10-06T11:23:03.581935+0000","last_clean":"2023-10-06T11:23:03.581935+0000","last_became_active":"2023-10-06T11:23:03.578821+0000","last_became_peered":"2023-10-06T11:23:03.578821+0000","last_unstale":"2023-10-06T11:23:03.581935+0000","last_undegraded":"2023-10-06T11:23:03.581935+0000","last_fullsized":"2023-10-06T11:23:03.581935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:26.243118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.580861+0000","last_change":"2023-10-06T11:23:03.578540+0000","last_active":"2023-10-06T11:23:03.580861+0000","last_peered":"2023-10-06T11:23:03.580861+0000","last_clean":"2023-10-06T11:23:03.580861+0000","last_became_active":"2023-10-06T11:23:03.578194+0000","last_became_peered":"2023-10-06T11:23:03.578194+0000","last_unstale":"2023-10-06T11:23:03.580861+0000","last_undegraded":"2023-10-06T11:23:03.580861+0000","last_fullsized":"2023-10-06T11:23:03.580861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:36:40.702516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592136+0000","last_change":"2023-10-06T11:23:03.582503+0000","last_active":"2023-10-06T11:23:03.592136+0000","last_peered":"2023-10-06T11:23:03.592136+0000","last_clean":"2023-10-06T11:23:03.592136+0000","last_became_active":"2023-10-06T11:23:03.582190+0000","last_became_peered":"2023-10-06T11:23:03.582190+0000","last_unstale":"2023-10-06T11:23:03.592136+0000","last_undegraded":"2023-10-06T11:23:03.592136+0000","last_fullsized":"2023-10-06T11:23:03.592136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:40:59.081090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574567+0000","last_change":"2023-10-06T11:23:03.572272+0000","last_active":"2023-10-06T11:23:03.574567+0000","last_peered":"2023-10-06T11:23:03.574567+0000","last_clean":"2023-10-06T11:23:03.574567+0000","last_became_active":"2023-10-06T11:23:03.571933+0000","last_became_peered":"2023-10-06T11:23:03.571933+0000","last_unstale":"2023-10-06T11:23:03.574567+0000","last_undegraded":"2023-10-06T11:23:03.574567+0000","last_fullsized":"2023-10-06T11:23:03.574567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:36:40.702516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585973+0000","last_change":"2023-10-06T11:23:03.583302+0000","last_active":"2023-10-06T11:23:03.585973+0000","last_peered":"2023-10-06T11:23:03.585973+0000","last_clean":"2023-10-06T11:23:03.585973+0000","last_became_active":"2023-10-06T11:23:03.582559+0000","last_became_peered":"2023-10-06T11:23:03.582559+0000","last_unstale":"2023-10-06T11:23:03.585973+0000","last_undegraded":"2023-10-06T11:23:03.585973+0000","last_fullsized":"2023-10-06T11:23:03.585973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:28.588171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585646+0000","last_change":"2023-10-06T11:23:03.585646+0000","last_active":"2023-10-06T11:23:03.585646+0000","last_peered":"2023-10-06T11:23:03.585646+0000","last_clean":"2023-10-06T11:23:03.585646+0000","last_became_active":"2023-10-06T11:23:03.584862+0000","last_became_peered":"2023-10-06T11:23:03.584862+0000","last_unstale":"2023-10-06T11:23:03.585646+0000","last_undegraded":"2023-10-06T11:23:03.585646+0000","last_fullsized":"2023-10-06T11:23:03.585646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593409+0000","last_change":"2023-10-06T11:23:03.586648+0000","last_active":"2023-10-06T11:23:03.593409+0000","last_peered":"2023-10-06T11:23:03.593409+0000","last_clean":"2023-10-06T11:23:03.593409+0000","last_became_active":"2023-10-06T11:23:03.586331+0000","last_became_peered":"2023-10-06T11:23:03.586331+0000","last_unstale":"2023-10-06T11:23:03.593409+0000","last_undegraded":"2023-10-06T11:23:03.593409+0000","last_fullsized":"2023-10-06T11:23:03.593409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:27.511696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591147+0000","last_change":"2023-10-06T11:23:03.588489+0000","last_active":"2023-10-06T11:23:03.591147+0000","last_peered":"2023-10-06T11:23:03.591147+0000","last_clean":"2023-10-06T11:23:03.591147+0000","last_became_active":"2023-10-06T11:23:03.588187+0000","last_became_peered":"2023-10-06T11:23:03.588187+0000","last_unstale":"2023-10-06T11:23:03.591147+0000","last_undegraded":"2023-10-06T11:23:03.591147+0000","last_fullsized":"2023-10-06T11:23:03.591147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:41.504246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581164+0000","last_change":"2023-10-06T11:23:03.576827+0000","last_active":"2023-10-06T11:23:03.581164+0000","last_peered":"2023-10-06T11:23:03.581164+0000","last_clean":"2023-10-06T11:23:03.581164+0000","last_became_active":"2023-10-06T11:23:03.576480+0000","last_became_peered":"2023-10-06T11:23:03.576480+0000","last_unstale":"2023-10-06T11:23:03.581164+0000","last_undegraded":"2023-10-06T11:23:03.581164+0000","last_fullsized":"2023-10-06T11:23:03.581164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:03.914121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.588818+0000","last_change":"2023-10-06T11:23:03.586173+0000","last_active":"2023-10-06T11:23:03.588818+0000","last_peered":"2023-10-06T11:23:03.588818+0000","last_clean":"2023-10-06T11:23:03.588818+0000","last_became_active":"2023-10-06T11:23:03.585842+0000","last_became_peered":"2023-10-06T11:23:03.585842+0000","last_unstale":"2023-10-06T11:23:03.588818+0000","last_undegraded":"2023-10-06T11:23:03.588818+0000","last_fullsized":"2023-10-06T11:23:03.588818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:51:14.805762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585863+0000","last_change":"2023-10-06T11:23:03.585863+0000","last_active":"2023-10-06T11:23:03.585863+0000","last_peered":"2023-10-06T11:23:03.585863+0000","last_clean":"2023-10-06T11:23:03.585863+0000","last_became_active":"2023-10-06T11:23:03.585255+0000","last_became_peered":"2023-10-06T11:23:03.585255+0000","last_unstale":"2023-10-06T11:23:03.585863+0000","last_undegraded":"2023-10-06T11:23:03.585863+0000","last_fullsized":"2023-10-06T11:23:03.585863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:41.423804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599269+0000","last_change":"2023-10-06T11:23:02.583346+0000","last_active":"2023-10-06T11:23:03.599269+0000","last_peered":"2023-10-06T11:23:03.599269+0000","last_clean":"2023-10-06T11:23:03.599269+0000","last_became_active":"2023-10-06T11:23:02.583017+0000","last_became_peered":"2023-10-06T11:23:02.583017+0000","last_unstale":"2023-10-06T11:23:03.599269+0000","last_undegraded":"2023-10-06T11:23:03.599269+0000","last_fullsized":"2023-10-06T11:23:03.599269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.256819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593440+0000","last_change":"2023-10-06T11:23:03.585930+0000","last_active":"2023-10-06T11:23:03.593440+0000","last_peered":"2023-10-06T11:23:03.593440+0000","last_clean":"2023-10-06T11:23:03.593440+0000","last_became_active":"2023-10-06T11:23:03.585678+0000","last_became_peered":"2023-10-06T11:23:03.585678+0000","last_unstale":"2023-10-06T11:23:03.593440+0000","last_undegraded":"2023-10-06T11:23:03.593440+0000","last_fullsized":"2023-10-06T11:23:03.593440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:17.348545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.598767+0000","last_change":"2023-10-06T11:23:02.578721+0000","last_active":"2023-10-06T11:23:03.598767+0000","last_peered":"2023-10-06T11:23:03.598767+0000","last_clean":"2023-10-06T11:23:03.598767+0000","last_became_active":"2023-10-06T11:23:02.578343+0000","last_became_peered":"2023-10-06T11:23:02.578343+0000","last_unstale":"2023-10-06T11:23:03.598767+0000","last_undegraded":"2023-10-06T11:23:03.598767+0000","last_fullsized":"2023-10-06T11:23:03.598767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:33.018879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.577151+0000","last_change":"2023-10-06T11:23:03.577151+0000","last_active":"2023-10-06T11:23:03.577151+0000","last_peered":"2023-10-06T11:23:03.577151+0000","last_clean":"2023-10-06T11:23:03.577151+0000","last_became_active":"2023-10-06T11:23:03.576625+0000","last_became_peered":"2023-10-06T11:23:03.576625+0000","last_unstale":"2023-10-06T11:23:03.577151+0000","last_undegraded":"2023-10-06T11:23:03.577151+0000","last_fullsized":"2023-10-06T11:23:03.577151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:35.666497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604761+0000","last_change":"2023-10-06T11:23:02.573591+0000","last_active":"2023-10-06T11:23:03.604761+0000","last_peered":"2023-10-06T11:23:03.604761+0000","last_clean":"2023-10-06T11:23:03.604761+0000","last_became_active":"2023-10-06T11:23:02.573273+0000","last_became_peered":"2023-10-06T11:23:02.573273+0000","last_unstale":"2023-10-06T11:23:03.604761+0000","last_undegraded":"2023-10-06T11:23:03.604761+0000","last_fullsized":"2023-10-06T11:23:03.604761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.582108+0000","last_change":"2023-10-06T11:23:03.582108+0000","last_active":"2023-10-06T11:23:03.582108+0000","last_peered":"2023-10-06T11:23:03.582108+0000","last_clean":"2023-10-06T11:23:03.582108+0000","last_became_active":"2023-10-06T11:23:03.581868+0000","last_became_peered":"2023-10-06T11:23:03.581868+0000","last_unstale":"2023-10-06T11:23:03.582108+0000","last_undegraded":"2023-10-06T11:23:03.582108+0000","last_fullsized":"2023-10-06T11:23:03.582108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:39:15.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600315+0000","last_change":"2023-10-06T11:23:02.585083+0000","last_active":"2023-10-06T11:23:03.600315+0000","last_peered":"2023-10-06T11:23:03.600315+0000","last_clean":"2023-10-06T11:23:03.600315+0000","last_became_active":"2023-10-06T11:23:02.584774+0000","last_became_peered":"2023-10-06T11:23:02.584774+0000","last_unstale":"2023-10-06T11:23:03.600315+0000","last_undegraded":"2023-10-06T11:23:03.600315+0000","last_fullsized":"2023-10-06T11:23:03.600315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.262517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585806+0000","last_change":"2023-10-06T11:23:03.584786+0000","last_active":"2023-10-06T11:23:03.585806+0000","last_peered":"2023-10-06T11:23:03.585806+0000","last_clean":"2023-10-06T11:23:03.585806+0000","last_became_active":"2023-10-06T11:23:03.584463+0000","last_became_peered":"2023-10-06T11:23:03.584463+0000","last_unstale":"2023-10-06T11:23:03.585806+0000","last_undegraded":"2023-10-06T11:23:03.585806+0000","last_fullsized":"2023-10-06T11:23:03.585806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:15:06.123004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600185+0000","last_change":"2023-10-06T11:23:02.574898+0000","last_active":"2023-10-06T11:23:03.600185+0000","last_peered":"2023-10-06T11:23:03.600185+0000","last_clean":"2023-10-06T11:23:03.600185+0000","last_became_active":"2023-10-06T11:23:02.574567+0000","last_became_peered":"2023-10-06T11:23:02.574567+0000","last_unstale":"2023-10-06T11:23:03.600185+0000","last_undegraded":"2023-10-06T11:23:03.600185+0000","last_fullsized":"2023-10-06T11:23:03.600185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:34.654820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589456+0000","last_change":"2023-10-06T11:23:03.589456+0000","last_active":"2023-10-06T11:23:03.589456+0000","last_peered":"2023-10-06T11:23:03.589456+0000","last_clean":"2023-10-06T11:23:03.589456+0000","last_became_active":"2023-10-06T11:23:03.587069+0000","last_became_peered":"2023-10-06T11:23:03.587069+0000","last_unstale":"2023-10-06T11:23:03.589456+0000","last_undegraded":"2023-10-06T11:23:03.589456+0000","last_fullsized":"2023-10-06T11:23:03.589456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:03.914121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600305+0000","last_change":"2023-10-06T11:23:02.571475+0000","last_active":"2023-10-06T11:23:03.600305+0000","last_peered":"2023-10-06T11:23:03.600305+0000","last_clean":"2023-10-06T11:23:03.600305+0000","last_became_active":"2023-10-06T11:23:02.571157+0000","last_became_peered":"2023-10-06T11:23:02.571157+0000","last_unstale":"2023-10-06T11:23:03.600305+0000","last_undegraded":"2023-10-06T11:23:03.600305+0000","last_fullsized":"2023-10-06T11:23:03.600305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:06.575163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589987+0000","last_change":"2023-10-06T11:23:03.589987+0000","last_active":"2023-10-06T11:23:03.589987+0000","last_peered":"2023-10-06T11:23:03.589987+0000","last_clean":"2023-10-06T11:23:03.589987+0000","last_became_active":"2023-10-06T11:23:03.589361+0000","last_became_peered":"2023-10-06T11:23:03.589361+0000","last_unstale":"2023-10-06T11:23:03.589987+0000","last_undegraded":"2023-10-06T11:23:03.589987+0000","last_fullsized":"2023-10-06T11:23:03.589987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:33.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585757+0000","last_change":"2023-10-06T11:23:02.583464+0000","last_active":"2023-10-06T11:23:02.585757+0000","last_peered":"2023-10-06T11:23:02.585757+0000","last_clean":"2023-10-06T11:23:02.585757+0000","last_became_active":"2023-10-06T11:23:02.582470+0000","last_became_peered":"2023-10-06T11:23:02.582470+0000","last_unstale":"2023-10-06T11:23:02.585757+0000","last_undegraded":"2023-10-06T11:23:02.585757+0000","last_fullsized":"2023-10-06T11:23:02.585757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:25.231442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.582006+0000","last_change":"2023-10-06T11:23:03.579545+0000","last_active":"2023-10-06T11:23:03.582006+0000","last_peered":"2023-10-06T11:23:03.582006+0000","last_clean":"2023-10-06T11:23:03.582006+0000","last_became_active":"2023-10-06T11:23:03.579186+0000","last_became_peered":"2023-10-06T11:23:03.579186+0000","last_unstale":"2023-10-06T11:23:03.582006+0000","last_undegraded":"2023-10-06T11:23:03.582006+0000","last_fullsized":"2023-10-06T11:23:03.582006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:46.920650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586019+0000","last_change":"2023-10-06T11:23:02.585174+0000","last_active":"2023-10-06T11:23:02.586019+0000","last_peered":"2023-10-06T11:23:02.586019+0000","last_clean":"2023-10-06T11:23:02.586019+0000","last_became_active":"2023-10-06T11:23:02.583075+0000","last_became_peered":"2023-10-06T11:23:02.583075+0000","last_unstale":"2023-10-06T11:23:02.586019+0000","last_undegraded":"2023-10-06T11:23:02.586019+0000","last_fullsized":"2023-10-06T11:23:02.586019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:25.231442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.601302+0000","last_change":"2023-10-06T11:23:03.599830+0000","last_active":"2023-10-06T11:23:03.601302+0000","last_peered":"2023-10-06T11:23:03.601302+0000","last_clean":"2023-10-06T11:23:03.601302+0000","last_became_active":"2023-10-06T11:23:03.599478+0000","last_became_peered":"2023-10-06T11:23:03.599478+0000","last_unstale":"2023-10-06T11:23:03.601302+0000","last_undegraded":"2023-10-06T11:23:03.601302+0000","last_fullsized":"2023-10-06T11:23:03.601302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:00.230481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.577772+0000","last_change":"2023-10-06T11:23:02.569019+0000","last_active":"2023-10-06T11:23:03.577772+0000","last_peered":"2023-10-06T11:23:03.577772+0000","last_clean":"2023-10-06T11:23:03.577772+0000","last_became_active":"2023-10-06T11:23:02.568670+0000","last_became_peered":"2023-10-06T11:23:02.568670+0000","last_unstale":"2023-10-06T11:23:03.577772+0000","last_undegraded":"2023-10-06T11:23:03.577772+0000","last_fullsized":"2023-10-06T11:23:03.577772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:41.553006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581029+0000","last_change":"2023-10-06T11:23:03.581029+0000","last_active":"2023-10-06T11:23:03.581029+0000","last_peered":"2023-10-06T11:23:03.581029+0000","last_clean":"2023-10-06T11:23:03.581029+0000","last_became_active":"2023-10-06T11:23:03.576455+0000","last_became_peered":"2023-10-06T11:23:03.576455+0000","last_unstale":"2023-10-06T11:23:03.581029+0000","last_undegraded":"2023-10-06T11:23:03.581029+0000","last_fullsized":"2023-10-06T11:23:03.581029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:59:07.586839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.584336+0000","last_change":"2023-10-06T11:23:02.580722+0000","last_active":"2023-10-06T11:23:02.584336+0000","last_peered":"2023-10-06T11:23:02.584336+0000","last_clean":"2023-10-06T11:23:02.584336+0000","last_became_active":"2023-10-06T11:23:02.580440+0000","last_became_peered":"2023-10-06T11:23:02.580440+0000","last_unstale":"2023-10-06T11:23:02.584336+0000","last_undegraded":"2023-10-06T11:23:02.584336+0000","last_fullsized":"2023-10-06T11:23:02.584336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:30.598999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.575167+0000","last_change":"2023-10-06T11:23:03.572491+0000","last_active":"2023-10-06T11:23:03.575167+0000","last_peered":"2023-10-06T11:23:03.575167+0000","last_clean":"2023-10-06T11:23:03.575167+0000","last_became_active":"2023-10-06T11:23:03.572133+0000","last_became_peered":"2023-10-06T11:23:03.572133+0000","last_unstale":"2023-10-06T11:23:03.575167+0000","last_undegraded":"2023-10-06T11:23:03.575167+0000","last_fullsized":"2023-10-06T11:23:03.575167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600068+0000","last_change":"2023-10-06T11:23:02.586384+0000","last_active":"2023-10-06T11:23:03.600068+0000","last_peered":"2023-10-06T11:23:03.600068+0000","last_clean":"2023-10-06T11:23:03.600068+0000","last_became_active":"2023-10-06T11:23:02.585802+0000","last_became_peered":"2023-10-06T11:23:02.585802+0000","last_unstale":"2023-10-06T11:23:03.600068+0000","last_undegraded":"2023-10-06T11:23:03.600068+0000","last_fullsized":"2023-10-06T11:23:03.600068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.191497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584094+0000","last_change":"2023-10-06T11:23:03.584094+0000","last_active":"2023-10-06T11:23:03.584094+0000","last_peered":"2023-10-06T11:23:03.584094+0000","last_clean":"2023-10-06T11:23:03.584094+0000","last_became_active":"2023-10-06T11:23:03.583806+0000","last_became_peered":"2023-10-06T11:23:03.583806+0000","last_unstale":"2023-10-06T11:23:03.584094+0000","last_undegraded":"2023-10-06T11:23:03.584094+0000","last_fullsized":"2023-10-06T11:23:03.584094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604717+0000","last_change":"2023-10-06T11:23:02.571591+0000","last_active":"2023-10-06T11:23:03.604717+0000","last_peered":"2023-10-06T11:23:03.604717+0000","last_clean":"2023-10-06T11:23:03.604717+0000","last_became_active":"2023-10-06T11:23:02.570952+0000","last_became_peered":"2023-10-06T11:23:02.570952+0000","last_unstale":"2023-10-06T11:23:03.604717+0000","last_undegraded":"2023-10-06T11:23:03.604717+0000","last_fullsized":"2023-10-06T11:23:03.604717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:47.697735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593969+0000","last_change":"2023-10-06T11:23:03.591947+0000","last_active":"2023-10-06T11:23:03.593969+0000","last_peered":"2023-10-06T11:23:03.593969+0000","last_clean":"2023-10-06T11:23:03.593969+0000","last_became_active":"2023-10-06T11:23:03.591616+0000","last_became_peered":"2023-10-06T11:23:03.591616+0000","last_unstale":"2023-10-06T11:23:03.593969+0000","last_undegraded":"2023-10-06T11:23:03.593969+0000","last_fullsized":"2023-10-06T11:23:03.593969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:31.610675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.604950+0000","last_change":"2023-10-06T11:23:02.582906+0000","last_active":"2023-10-06T11:23:03.604950+0000","last_peered":"2023-10-06T11:23:03.604950+0000","last_clean":"2023-10-06T11:23:03.604950+0000","last_became_active":"2023-10-06T11:23:02.582575+0000","last_became_peered":"2023-10-06T11:23:02.582575+0000","last_unstale":"2023-10-06T11:23:03.604950+0000","last_undegraded":"2023-10-06T11:23:03.604950+0000","last_fullsized":"2023-10-06T11:23:03.604950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:13.191497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.589547+0000","last_change":"2023-10-06T11:23:03.589547+0000","last_active":"2023-10-06T11:23:03.589547+0000","last_peered":"2023-10-06T11:23:03.589547+0000","last_clean":"2023-10-06T11:23:03.589547+0000","last_became_active":"2023-10-06T11:23:03.586698+0000","last_became_peered":"2023-10-06T11:23:03.586698+0000","last_unstale":"2023-10-06T11:23:03.589547+0000","last_undegraded":"2023-10-06T11:23:03.589547+0000","last_fullsized":"2023-10-06T11:23:03.589547+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586980+0000","last_change":"2023-10-06T11:23:02.580420+0000","last_active":"2023-10-06T11:23:02.586980+0000","last_peered":"2023-10-06T11:23:02.586980+0000","last_clean":"2023-10-06T11:23:02.586980+0000","last_became_active":"2023-10-06T11:23:02.580129+0000","last_became_peered":"2023-10-06T11:23:02.580129+0000","last_unstale":"2023-10-06T11:23:02.586980+0000","last_undegraded":"2023-10-06T11:23:02.586980+0000","last_fullsized":"2023-10-06T11:23:02.586980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:43:20.256819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584821+0000","last_change":"2023-10-06T11:23:03.580096+0000","last_active":"2023-10-06T11:23:03.584821+0000","last_peered":"2023-10-06T11:23:03.584821+0000","last_clean":"2023-10-06T11:23:03.584821+0000","last_became_active":"2023-10-06T11:23:03.579809+0000","last_became_peered":"2023-10-06T11:23:03.579809+0000","last_unstale":"2023-10-06T11:23:03.584821+0000","last_undegraded":"2023-10-06T11:23:03.584821+0000","last_fullsized":"2023-10-06T11:23:03.584821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:02.622701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.564983+0000","last_change":"2023-10-06T11:23:02.578903+0000","last_active":"2023-10-06T11:23:03.564983+0000","last_peered":"2023-10-06T11:23:03.564983+0000","last_clean":"2023-10-06T11:23:03.564983+0000","last_became_active":"2023-10-06T11:23:02.578571+0000","last_became_peered":"2023-10-06T11:23:02.578571+0000","last_unstale":"2023-10-06T11:23:03.564983+0000","last_undegraded":"2023-10-06T11:23:03.564983+0000","last_fullsized":"2023-10-06T11:23:03.564983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:05:23.914766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.581721+0000","last_change":"2023-10-06T11:23:03.581721+0000","last_active":"2023-10-06T11:23:03.581721+0000","last_peered":"2023-10-06T11:23:03.581721+0000","last_clean":"2023-10-06T11:23:03.581721+0000","last_became_active":"2023-10-06T11:23:03.581501+0000","last_became_peered":"2023-10-06T11:23:03.581501+0000","last_unstale":"2023-10-06T11:23:03.581721+0000","last_undegraded":"2023-10-06T11:23:03.581721+0000","last_fullsized":"2023-10-06T11:23:03.581721+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:18.747941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.607932+0000","last_change":"2023-10-06T11:23:02.579294+0000","last_active":"2023-10-06T11:23:03.607932+0000","last_peered":"2023-10-06T11:23:03.607932+0000","last_clean":"2023-10-06T11:23:03.607932+0000","last_became_active":"2023-10-06T11:23:02.578991+0000","last_became_peered":"2023-10-06T11:23:02.578991+0000","last_unstale":"2023-10-06T11:23:03.607932+0000","last_undegraded":"2023-10-06T11:23:03.607932+0000","last_fullsized":"2023-10-06T11:23:03.607932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:56.297774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.578913+0000","last_change":"2023-10-06T11:23:03.576996+0000","last_active":"2023-10-06T11:23:03.578913+0000","last_peered":"2023-10-06T11:23:03.578913+0000","last_clean":"2023-10-06T11:23:03.578913+0000","last_became_active":"2023-10-06T11:23:03.576437+0000","last_became_peered":"2023-10-06T11:23:03.576437+0000","last_unstale":"2023-10-06T11:23:03.578913+0000","last_undegraded":"2023-10-06T11:23:03.578913+0000","last_fullsized":"2023-10-06T11:23:03.578913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:06:36.098524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.602299+0000","last_change":"2023-10-06T11:23:02.573627+0000","last_active":"2023-10-06T11:23:03.602299+0000","last_peered":"2023-10-06T11:23:03.602299+0000","last_clean":"2023-10-06T11:23:03.602299+0000","last_became_active":"2023-10-06T11:23:02.573286+0000","last_became_peered":"2023-10-06T11:23:02.573286+0000","last_unstale":"2023-10-06T11:23:03.602299+0000","last_undegraded":"2023-10-06T11:23:03.602299+0000","last_fullsized":"2023-10-06T11:23:03.602299+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:51.056271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.591711+0000","last_change":"2023-10-06T11:23:03.591711+0000","last_active":"2023-10-06T11:23:03.591711+0000","last_peered":"2023-10-06T11:23:03.591711+0000","last_clean":"2023-10-06T11:23:03.591711+0000","last_became_active":"2023-10-06T11:23:03.591353+0000","last_became_peered":"2023-10-06T11:23:03.591353+0000","last_unstale":"2023-10-06T11:23:03.591711+0000","last_undegraded":"2023-10-06T11:23:03.591711+0000","last_fullsized":"2023-10-06T11:23:03.591711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:44.048834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.605132+0000","last_change":"2023-10-06T11:23:02.575738+0000","last_active":"2023-10-06T11:23:03.605132+0000","last_peered":"2023-10-06T11:23:03.605132+0000","last_clean":"2023-10-06T11:23:03.605132+0000","last_became_active":"2023-10-06T11:23:02.575192+0000","last_became_peered":"2023-10-06T11:23:02.575192+0000","last_unstale":"2023-10-06T11:23:03.605132+0000","last_undegraded":"2023-10-06T11:23:03.605132+0000","last_fullsized":"2023-10-06T11:23:03.605132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:43.037158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.572180+0000","last_change":"2023-10-06T11:23:03.570835+0000","last_active":"2023-10-06T11:23:03.572180+0000","last_peered":"2023-10-06T11:23:03.572180+0000","last_clean":"2023-10-06T11:23:03.572180+0000","last_became_active":"2023-10-06T11:23:03.570489+0000","last_became_peered":"2023-10-06T11:23:03.570489+0000","last_unstale":"2023-10-06T11:23:03.572180+0000","last_undegraded":"2023-10-06T11:23:03.572180+0000","last_fullsized":"2023-10-06T11:23:03.572180+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:02.611041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.574862+0000","last_change":"2023-10-06T11:23:02.582228+0000","last_active":"2023-10-06T11:23:03.574862+0000","last_peered":"2023-10-06T11:23:03.574862+0000","last_clean":"2023-10-06T11:23:03.574862+0000","last_became_active":"2023-10-06T11:23:02.581935+0000","last_became_peered":"2023-10-06T11:23:02.581935+0000","last_unstale":"2023-10-06T11:23:03.574862+0000","last_undegraded":"2023-10-06T11:23:03.574862+0000","last_fullsized":"2023-10-06T11:23:03.574862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:30.598999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.593406+0000","last_change":"2023-10-06T11:23:03.583310+0000","last_active":"2023-10-06T11:23:03.593406+0000","last_peered":"2023-10-06T11:23:03.593406+0000","last_clean":"2023-10-06T11:23:03.593406+0000","last_became_active":"2023-10-06T11:23:03.582899+0000","last_became_peered":"2023-10-06T11:23:03.582899+0000","last_unstale":"2023-10-06T11:23:03.593406+0000","last_undegraded":"2023-10-06T11:23:03.593406+0000","last_fullsized":"2023-10-06T11:23:03.593406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:43.200957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.563917+0000","last_change":"2023-10-06T11:23:02.577006+0000","last_active":"2023-10-06T11:23:03.563917+0000","last_peered":"2023-10-06T11:23:03.563917+0000","last_clean":"2023-10-06T11:23:03.563917+0000","last_became_active":"2023-10-06T11:23:02.576588+0000","last_became_peered":"2023-10-06T11:23:02.576588+0000","last_unstale":"2023-10-06T11:23:03.563917+0000","last_undegraded":"2023-10-06T11:23:03.563917+0000","last_fullsized":"2023-10-06T11:23:03.563917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:23.201591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.575338+0000","last_change":"2023-10-06T11:23:03.572370+0000","last_active":"2023-10-06T11:23:03.575338+0000","last_peered":"2023-10-06T11:23:03.575338+0000","last_clean":"2023-10-06T11:23:03.575338+0000","last_became_active":"2023-10-06T11:23:03.572053+0000","last_became_peered":"2023-10-06T11:23:03.572053+0000","last_unstale":"2023-10-06T11:23:03.575338+0000","last_undegraded":"2023-10-06T11:23:03.575338+0000","last_fullsized":"2023-10-06T11:23:03.575338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:29.461366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600291+0000","last_change":"2023-10-06T11:23:02.573151+0000","last_active":"2023-10-06T11:23:03.600291+0000","last_peered":"2023-10-06T11:23:03.600291+0000","last_clean":"2023-10-06T11:23:03.600291+0000","last_became_active":"2023-10-06T11:23:02.572186+0000","last_became_peered":"2023-10-06T11:23:02.572186+0000","last_unstale":"2023-10-06T11:23:03.600291+0000","last_undegraded":"2023-10-06T11:23:03.600291+0000","last_fullsized":"2023-10-06T11:23:03.600291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.262517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.594176+0000","last_change":"2023-10-06T11:23:03.594176+0000","last_active":"2023-10-06T11:23:03.594176+0000","last_peered":"2023-10-06T11:23:03.594176+0000","last_clean":"2023-10-06T11:23:03.594176+0000","last_became_active":"2023-10-06T11:23:03.593888+0000","last_became_peered":"2023-10-06T11:23:03.593888+0000","last_unstale":"2023-10-06T11:23:03.594176+0000","last_undegraded":"2023-10-06T11:23:03.594176+0000","last_fullsized":"2023-10-06T11:23:03.594176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:43.080095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.583902+0000","last_change":"2023-10-06T11:23:02.581847+0000","last_active":"2023-10-06T11:23:02.583902+0000","last_peered":"2023-10-06T11:23:02.583902+0000","last_clean":"2023-10-06T11:23:02.583902+0000","last_became_active":"2023-10-06T11:23:02.581488+0000","last_became_peered":"2023-10-06T11:23:02.581488+0000","last_unstale":"2023-10-06T11:23:02.583902+0000","last_undegraded":"2023-10-06T11:23:02.583902+0000","last_fullsized":"2023-10-06T11:23:02.583902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.579355+0000","last_change":"2023-10-06T11:23:03.578701+0000","last_active":"2023-10-06T11:23:03.579355+0000","last_peered":"2023-10-06T11:23:03.579355+0000","last_clean":"2023-10-06T11:23:03.579355+0000","last_became_active":"2023-10-06T11:23:03.578413+0000","last_became_peered":"2023-10-06T11:23:03.578413+0000","last_unstale":"2023-10-06T11:23:03.579355+0000","last_undegraded":"2023-10-06T11:23:03.579355+0000","last_fullsized":"2023-10-06T11:23:03.579355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:24.416766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.584483+0000","last_change":"2023-10-06T11:23:02.581460+0000","last_active":"2023-10-06T11:23:02.584483+0000","last_peered":"2023-10-06T11:23:02.584483+0000","last_clean":"2023-10-06T11:23:02.584483+0000","last_became_active":"2023-10-06T11:23:02.580546+0000","last_became_peered":"2023-10-06T11:23:02.580546+0000","last_unstale":"2023-10-06T11:23:02.584483+0000","last_undegraded":"2023-10-06T11:23:02.584483+0000","last_fullsized":"2023-10-06T11:23:02.584483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:45.908974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.595176+0000","last_change":"2023-10-06T11:23:03.595176+0000","last_active":"2023-10-06T11:23:03.595176+0000","last_peered":"2023-10-06T11:23:03.595176+0000","last_clean":"2023-10-06T11:23:03.595176+0000","last_became_active":"2023-10-06T11:23:03.593091+0000","last_became_peered":"2023-10-06T11:23:03.593091+0000","last_unstale":"2023-10-06T11:23:03.595176+0000","last_undegraded":"2023-10-06T11:23:03.595176+0000","last_fullsized":"2023-10-06T11:23:03.595176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:54.497474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585676+0000","last_change":"2023-10-06T11:23:02.578781+0000","last_active":"2023-10-06T11:23:02.585676+0000","last_peered":"2023-10-06T11:23:02.585676+0000","last_clean":"2023-10-06T11:23:02.585676+0000","last_became_active":"2023-10-06T11:23:02.578475+0000","last_became_peered":"2023-10-06T11:23:02.578475+0000","last_unstale":"2023-10-06T11:23:02.585676+0000","last_undegraded":"2023-10-06T11:23:02.585676+0000","last_fullsized":"2023-10-06T11:23:02.585676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:08.736830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.585748+0000","last_change":"2023-10-06T11:23:03.583220+0000","last_active":"2023-10-06T11:23:03.585748+0000","last_peered":"2023-10-06T11:23:03.585748+0000","last_clean":"2023-10-06T11:23:03.585748+0000","last_became_active":"2023-10-06T11:23:03.582786+0000","last_became_peered":"2023-10-06T11:23:03.582786+0000","last_unstale":"2023-10-06T11:23:03.585748+0000","last_undegraded":"2023-10-06T11:23:03.585748+0000","last_fullsized":"2023-10-06T11:23:03.585748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:02.622701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.585316+0000","last_change":"2023-10-06T11:23:02.583486+0000","last_active":"2023-10-06T11:23:02.585316+0000","last_peered":"2023-10-06T11:23:02.585316+0000","last_clean":"2023-10-06T11:23:02.585316+0000","last_became_active":"2023-10-06T11:23:02.583093+0000","last_became_peered":"2023-10-06T11:23:02.583093+0000","last_unstale":"2023-10-06T11:23:02.585316+0000","last_undegraded":"2023-10-06T11:23:02.585316+0000","last_fullsized":"2023-10-06T11:23:02.585316+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:16.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.567567+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:03.567567+0000","last_peered":"2023-10-06T11:23:03.567567+0000","last_clean":"2023-10-06T11:23:03.567567+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:03.567567+0000","last_undegraded":"2023-10-06T11:23:03.567567+0000","last_fullsized":"2023-10-06T11:23:03.567567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.584732+0000","last_change":"2023-10-06T11:23:03.584732+0000","last_active":"2023-10-06T11:23:03.584732+0000","last_peered":"2023-10-06T11:23:03.584732+0000","last_clean":"2023-10-06T11:23:03.584732+0000","last_became_active":"2023-10-06T11:23:03.584369+0000","last_became_peered":"2023-10-06T11:23:03.584369+0000","last_unstale":"2023-10-06T11:23:03.584732+0000","last_undegraded":"2023-10-06T11:23:03.584732+0000","last_fullsized":"2023-10-06T11:23:03.584732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:59.274194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.586447+0000","last_change":"2023-10-06T11:23:02.585146+0000","last_active":"2023-10-06T11:23:02.586447+0000","last_peered":"2023-10-06T11:23:02.586447+0000","last_clean":"2023-10-06T11:23:02.586447+0000","last_became_active":"2023-10-06T11:23:02.584815+0000","last_became_peered":"2023-10-06T11:23:02.584815+0000","last_unstale":"2023-10-06T11:23:02.586447+0000","last_undegraded":"2023-10-06T11:23:02.586447+0000","last_fullsized":"2023-10-06T11:23:02.586447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:42.189280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.566539+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:03.566539+0000","last_peered":"2023-10-06T11:23:03.566539+0000","last_clean":"2023-10-06T11:23:03.566539+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:03.566539+0000","last_undegraded":"2023-10-06T11:23:03.566539+0000","last_fullsized":"2023-10-06T11:23:03.566539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.586094+0000","last_change":"2023-10-06T11:23:03.580448+0000","last_active":"2023-10-06T11:23:03.586094+0000","last_peered":"2023-10-06T11:23:03.586094+0000","last_clean":"2023-10-06T11:23:03.586094+0000","last_became_active":"2023-10-06T11:23:03.580126+0000","last_became_peered":"2023-10-06T11:23:03.580126+0000","last_unstale":"2023-10-06T11:23:03.586094+0000","last_undegraded":"2023-10-06T11:23:03.586094+0000","last_fullsized":"2023-10-06T11:23:03.586094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:19.535711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.579947+0000","last_change":"2023-10-06T11:23:02.573383+0000","last_active":"2023-10-06T11:23:03.579947+0000","last_peered":"2023-10-06T11:23:03.579947+0000","last_clean":"2023-10-06T11:23:03.579947+0000","last_became_active":"2023-10-06T11:23:02.573018+0000","last_became_peered":"2023-10-06T11:23:02.573018+0000","last_unstale":"2023-10-06T11:23:03.579947+0000","last_undegraded":"2023-10-06T11:23:03.579947+0000","last_fullsized":"2023-10-06T11:23:03.579947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:30.773636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.572622+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:03.572622+0000","last_peered":"2023-10-06T11:23:03.572622+0000","last_clean":"2023-10-06T11:23:03.572622+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:03.572622+0000","last_undegraded":"2023-10-06T11:23:03.572622+0000","last_fullsized":"2023-10-06T11:23:03.572622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.587681+0000","last_change":"2023-10-06T11:23:03.586717+0000","last_active":"2023-10-06T11:23:03.587681+0000","last_peered":"2023-10-06T11:23:03.587681+0000","last_clean":"2023-10-06T11:23:03.587681+0000","last_became_active":"2023-10-06T11:23:03.586428+0000","last_became_peered":"2023-10-06T11:23:03.586428+0000","last_unstale":"2023-10-06T11:23:03.587681+0000","last_undegraded":"2023-10-06T11:23:03.587681+0000","last_fullsized":"2023-10-06T11:23:03.587681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:54:22.209111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.587162+0000","last_change":"2023-10-06T11:23:02.583977+0000","last_active":"2023-10-06T11:23:02.587162+0000","last_peered":"2023-10-06T11:23:02.587162+0000","last_clean":"2023-10-06T11:23:02.587162+0000","last_became_active":"2023-10-06T11:23:02.583629+0000","last_became_peered":"2023-10-06T11:23:02.583629+0000","last_unstale":"2023-10-06T11:23:02.587162+0000","last_undegraded":"2023-10-06T11:23:02.587162+0000","last_fullsized":"2023-10-06T11:23:02.587162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:27.024345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.564127+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:03.564127+0000","last_peered":"2023-10-06T11:23:03.564127+0000","last_clean":"2023-10-06T11:23:03.564127+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:03.564127+0000","last_undegraded":"2023-10-06T11:23:03.564127+0000","last_fullsized":"2023-10-06T11:23:03.564127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.578331+0000","last_change":"2023-10-06T11:23:03.578331+0000","last_active":"2023-10-06T11:23:03.578331+0000","last_peered":"2023-10-06T11:23:03.578331+0000","last_clean":"2023-10-06T11:23:03.578331+0000","last_became_active":"2023-10-06T11:23:03.578001+0000","last_became_peered":"2023-10-06T11:23:03.578001+0000","last_unstale":"2023-10-06T11:23:03.578331+0000","last_undegraded":"2023-10-06T11:23:03.578331+0000","last_fullsized":"2023-10-06T11:23:03.578331+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:54.497474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.582774+0000","last_change":"2023-10-06T11:23:02.581981+0000","last_active":"2023-10-06T11:23:02.582774+0000","last_peered":"2023-10-06T11:23:02.582774+0000","last_clean":"2023-10-06T11:23:02.582774+0000","last_became_active":"2023-10-06T11:23:02.581679+0000","last_became_peered":"2023-10-06T11:23:02.581679+0000","last_unstale":"2023-10-06T11:23:02.582774+0000","last_undegraded":"2023-10-06T11:23:02.582774+0000","last_fullsized":"2023-10-06T11:23:02.582774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:26.500019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.570952+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:02.570952+0000","last_peered":"2023-10-06T11:23:02.570952+0000","last_clean":"2023-10-06T11:23:02.570952+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:02.570952+0000","last_undegraded":"2023-10-06T11:23:02.570952+0000","last_fullsized":"2023-10-06T11:23:02.570952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.573217+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:02.573217+0000","last_peered":"2023-10-06T11:23:02.573217+0000","last_clean":"2023-10-06T11:23:02.573217+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:02.573217+0000","last_undegraded":"2023-10-06T11:23:02.573217+0000","last_fullsized":"2023-10-06T11:23:02.573217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.592053+0000","last_change":"2023-10-06T11:23:03.592053+0000","last_active":"2023-10-06T11:23:03.592053+0000","last_peered":"2023-10-06T11:23:03.592053+0000","last_clean":"2023-10-06T11:23:03.592053+0000","last_became_active":"2023-10-06T11:23:03.590737+0000","last_became_peered":"2023-10-06T11:23:03.590737+0000","last_unstale":"2023-10-06T11:23:03.592053+0000","last_undegraded":"2023-10-06T11:23:03.592053+0000","last_fullsized":"2023-10-06T11:23:03.592053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:34.030555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.587582+0000","last_change":"2023-10-06T11:23:02.586417+0000","last_active":"2023-10-06T11:23:02.587582+0000","last_peered":"2023-10-06T11:23:02.587582+0000","last_clean":"2023-10-06T11:23:02.587582+0000","last_became_active":"2023-10-06T11:23:02.586131+0000","last_became_peered":"2023-10-06T11:23:02.586131+0000","last_unstale":"2023-10-06T11:23:02.587582+0000","last_undegraded":"2023-10-06T11:23:02.587582+0000","last_fullsized":"2023-10-06T11:23:02.587582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.003668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.574627+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:02.574627+0000","last_peered":"2023-10-06T11:23:02.574627+0000","last_clean":"2023-10-06T11:23:02.574627+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:02.574627+0000","last_undegraded":"2023-10-06T11:23:02.574627+0000","last_fullsized":"2023-10-06T11:23:02.574627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.596975+0000","last_change":"2023-10-06T11:23:03.593178+0000","last_active":"2023-10-06T11:23:03.596975+0000","last_peered":"2023-10-06T11:23:03.596975+0000","last_clean":"2023-10-06T11:23:03.596975+0000","last_became_active":"2023-10-06T11:23:03.592800+0000","last_became_peered":"2023-10-06T11:23:03.592800+0000","last_unstale":"2023-10-06T11:23:03.596975+0000","last_undegraded":"2023-10-06T11:23:03.596975+0000","last_fullsized":"2023-10-06T11:23:03.596975+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:43.080095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.565130+0000","last_change":"2023-10-06T11:23:02.571930+0000","last_active":"2023-10-06T11:23:03.565130+0000","last_peered":"2023-10-06T11:23:03.565130+0000","last_clean":"2023-10-06T11:23:03.565130+0000","last_became_active":"2023-10-06T11:23:02.571593+0000","last_became_peered":"2023-10-06T11:23:02.571593+0000","last_unstale":"2023-10-06T11:23:03.565130+0000","last_undegraded":"2023-10-06T11:23:03.565130+0000","last_fullsized":"2023-10-06T11:23:03.565130+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.003668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.569581+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:02.569581+0000","last_peered":"2023-10-06T11:23:02.569581+0000","last_clean":"2023-10-06T11:23:02.569581+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:02.569581+0000","last_undegraded":"2023-10-06T11:23:02.569581+0000","last_fullsized":"2023-10-06T11:23:02.569581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.599137+0000","last_change":"2023-10-06T11:23:03.599137+0000","last_active":"2023-10-06T11:23:03.599137+0000","last_peered":"2023-10-06T11:23:03.599137+0000","last_clean":"2023-10-06T11:23:03.599137+0000","last_became_active":"2023-10-06T11:23:03.598829+0000","last_became_peered":"2023-10-06T11:23:03.598829+0000","last_unstale":"2023-10-06T11:23:03.599137+0000","last_undegraded":"2023-10-06T11:23:03.599137+0000","last_fullsized":"2023-10-06T11:23:03.599137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:02.542240+0000","last_clean_scrub_stamp":"2023-10-06T11:23:02.542240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:54.566255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T11:23:03.600001+0000","last_change":"2023-10-06T11:23:02.582219+0000","last_active":"2023-10-06T11:23:03.600001+0000","last_peered":"2023-10-06T11:23:03.600001+0000","last_clean":"2023-10-06T11:23:03.600001+0000","last_became_active":"2023-10-06T11:23:02.581848+0000","last_became_peered":"2023-10-06T11:23:02.581848+0000","last_unstale":"2023-10-06T11:23:03.600001+0000","last_undegraded":"2023-10-06T11:23:03.600001+0000","last_fullsized":"2023-10-06T11:23:03.600001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:01.530563+0000","last_clean_scrub_stamp":"2023-10-06T11:23:01.530563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:45.908974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T11:23:02.571471+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:02.571471+0000","last_peered":"2023-10-06T11:23:02.571471+0000","last_clean":"2023-10-06T11:23:02.571471+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:02.571471+0000","last_undegraded":"2023-10-06T11:23:02.571471+0000","last_fullsized":"2023-10-06T11:23:02.571471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1036288,"data_stored":785500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.98399999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0529999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.94899999999999995}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0820000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301820,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070020,"statfs":{"total":96636764160,"available":96327700480,"internally_reserved":0,"allocated":446464,"data_stored":195686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85299999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.0129999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.266},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.2250000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.6850000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.573},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.617},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.738}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":442368,"data_stored":195112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.746},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.02}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.079},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.87},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.21}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301836,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070004,"statfs":{"total":96636764160,"available":96327684096,"internally_reserved":0,"allocated":462848,"data_stored":203539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.256},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.93300000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.998},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":1.1699999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.286},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.125}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":1.385},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.351}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.329}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301836,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070004,"statfs":{"total":96636764160,"available":96327684096,"internally_reserved":0,"allocated":462848,"data_stored":202613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.874},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1679999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.044}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.117},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301840,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070000,"statfs":{"total":96636764160,"available":96327680000,"internally_reserved":0,"allocated":466944,"data_stored":203561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.0629999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":1036288,"data_stored":785480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.998},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":442368,"data_stored":195076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.93200000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1899999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.163},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0900000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0649999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.137}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:07.928 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:23:07.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:23:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.043+0000 7f5336194700 1 Processor -- start 2023-10-06T11:23:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.043+0000 7f5336194700 1 -- start start 2023-10-06T11:23:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f53300b41e0 0x7f53300b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f533006de10 0x7f5330159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53300b1190 con 0x7f53300b3e10 2023-10-06T11:23:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53300b0f80 con 0x7f533006de10 2023-10-06T11:23:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53300b1470 con 0x7f53300b41e0 2023-10-06T11:23:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f533006de10 0x7f5330159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f533006de10 0x7f5330159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54202/0 (socket says 172.21.15.7:54202) 2023-10-06T11:23:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 -- 172.21.15.7:0/730688131 learned_addr learned my addr 172.21.15.7:0/730688131 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5334991700 1 --2- 172.21.15.7:0/730688131 >> v2:172.21.15.50:3300/0 conn(0x7f53300b41e0 0x7f53300b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 -- 172.21.15.7:0/730688131 >> v2:172.21.15.50:3300/0 conn(0x7f53300b41e0 msgr2=0x7f53300b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 --2- 172.21.15.7:0/730688131 >> v2:172.21.15.50:3300/0 conn(0x7f53300b41e0 0x7f53300b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 -- 172.21.15.7:0/730688131 >> v1:172.21.15.50:6789/0 conn(0x7f53300b3e10 legacy=0x7f533006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 -- 172.21.15.7:0/730688131 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53300b1750 con 0x7f533006de10 2023-10-06T11:23:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5335993700 1 --2- 172.21.15.7:0/730688131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f5330159870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f53300b0a90 tx=0x7f5328009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676791bbe3bd2fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5323fff700 1 -- 172.21.15.7:0/730688131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532800e070 con 0x7f533006de10 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5323fff700 1 -- 172.21.15.7:0/730688131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532800ba20 con 0x7f533006de10 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5323fff700 1 -- 172.21.15.7:0/730688131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280092e0 con 0x7f533006de10 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- 172.21.15.7:0/730688131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 msgr2=0x7f5330159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 --2- 172.21.15.7:0/730688131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f5330159870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f53300b0a90 tx=0x7f5328009840 comp rx=0 tx=0).stop 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- 172.21.15.7:0/730688131 shutdown_connections 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 --2- 172.21.15.7:0/730688131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f5330159870 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 --2- 172.21.15.7:0/730688131 >> v2:172.21.15.50:3300/0 conn(0x7f53300b41e0 0x7f53300b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- 172.21.15.7:0/730688131 >> 172.21.15.7:0/730688131 conn(0x7f53300cfd00 msgr2=0x7f53300d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- 172.21.15.7:0/730688131 shutdown_connections 2023-10-06T11:23:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.047+0000 7f5336194700 1 -- 172.21.15.7:0/730688131 wait complete. 2023-10-06T11:23:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 Processor -- start 2023-10-06T11:23:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 -- start start 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f533014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53300b3e10 0x7f533006a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 0x7f533006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53300cfa00 con 0x7f533006de10 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301696b0 con 0x7f53300b3e10 2023-10-06T11:23:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5336194700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533016a230 con 0x7f53300b41e0 2023-10-06T11:23:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5335192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f533014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5335192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f533014cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54218/0 (socket says 172.21.15.7:54218) 2023-10-06T11:23:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5335192700 1 -- 172.21.15.7:0/2441970238 learned_addr learned my addr 172.21.15.7:0/2441970238 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5335993700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 0x7f533006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.051+0000 7f5334991700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53300b3e10 0x7f533006a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 -- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53300b3e10 msgr2=0x7f533006a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53300b3e10 0x7f533006a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 -- 172.21.15.7:0/2441970238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 msgr2=0x7f533014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f533014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 -- 172.21.15.7:0/2441970238 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5328009ae0 con 0x7f53300b41e0 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5335993700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 0x7f533006ab20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5328015000 tx=0x7f5328018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328008a80 con 0x7f53300b41e0 2023-10-06T11:23:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328008c20 con 0x7f53300b41e0 2023-10-06T11:23:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533016a6c0 con 0x7f53300b41e0 2023-10-06T11:23:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328026460 con 0x7f53300b41e0 2023-10-06T11:23:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5330169be0 con 0x7f53300b41e0 2023-10-06T11:23:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53300b22f0 con 0x7f53300b41e0 2023-10-06T11:23:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.055+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5328003040 con 0x7f53300b41e0 2023-10-06T11:23:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.059+0000 7f53217fa700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 0x7f5314062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.059+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f532807e980 con 0x7f53300b41e0 2023-10-06T11:23:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.059+0000 7f5335192700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 0x7f5314062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.059+0000 7f5335192700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 0x7f5314062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f533006b2e0 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.071+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f53300b22f0 con 0x7f53300b41e0 2023-10-06T11:23:08.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.287+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f53300b22f0 con 0x7f53300b41e0 2023-10-06T11:23:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f53217fa700 1 -- 172.21.15.7:0/2441970238 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f53300b22f0 con 0x7f53300b41e0 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 msgr2=0x7f5314062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 0x7f5314062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f533006b2e0 tx=0x7f5324009040 comp rx=0 tx=0).stop 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 msgr2=0x7f533006ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 0x7f533006ab20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5328015000 tx=0x7f5328018000 comp rx=0 tx=0).stop 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 shutdown_connections 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f5314060550 0x7f5314062a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f53300b41e0 0x7f533006ab20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53300b3e10 0x7f533006a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 --2- 172.21.15.7:0/2441970238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f533006de10 0x7f533014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 >> 172.21.15.7:0/2441970238 conn(0x7f53300cfd00 msgr2=0x7f5330158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 shutdown_connections 2023-10-06T11:23:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.299+0000 7f5336194700 1 -- 172.21.15.7:0/2441970238 wait complete. 2023-10-06T11:23:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2023-10-06T11:23:08.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 Processor -- start 2023-10-06T11:23:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 -- start start 2023-10-06T11:23:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 0x7fb2340c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2340c9950 0x7fb234067e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb234157160 con 0x7fb234068370 2023-10-06T11:23:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2340687d0 con 0x7fb2340c9950 2023-10-06T11:23:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb238b7c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340ca3b0 con 0x7fb2340ac070 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2340c9950 0x7fb234067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb233fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb234068370 0x7fb2340cc980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49746/0 (socket says 172.21.15.7:49746) 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb233fff700 1 -- 172.21.15.7:0/1589586764 learned_addr learned my addr 172.21.15.7:0/1589586764 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2337fe700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 0x7fb2340c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 -- 172.21.15.7:0/1589586764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 msgr2=0x7fb2340c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 0x7fb2340c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 -- 172.21.15.7:0/1589586764 >> v1:172.21.15.7:6789/0 conn(0x7fb234068370 legacy=0x7fb2340cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 -- 172.21.15.7:0/1589586764 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2340ac6d0 con 0x7fb2340c9950 2023-10-06T11:23:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2327fc700 1 -- 172.21.15.7:0/1589586764 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173641868 0 0) 0x7fb234157160 con 0x7fb234068370 2023-10-06T11:23:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2337fe700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 0x7fb2340c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb232ffd700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340c9950 0x7fb234067e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0098e0 tx=0x7fb21c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369f12da30a1fdfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2327fc700 1 -- 172.21.15.7:0/1589586764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c009e80 con 0x7fb2340c9950 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2327fc700 1 -- 172.21.15.7:0/1589586764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00b860 con 0x7fb2340c9950 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.419+0000 7fb2327fc700 1 -- 172.21.15.7:0/1589586764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00bbe0 con 0x7fb2340c9950 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- 172.21.15.7:0/1589586764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340c9950 msgr2=0x7fb234067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340c9950 0x7fb234067e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0098e0 tx=0x7fb21c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- 172.21.15.7:0/1589586764 shutdown_connections 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340c9950 0x7fb234067e30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 --2- 172.21.15.7:0/1589586764 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340ac070 0x7fb2340c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- 172.21.15.7:0/1589586764 >> 172.21.15.7:0/1589586764 conn(0x7fb23406f380 msgr2=0x7fb23406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- 172.21.15.7:0/1589586764 shutdown_connections 2023-10-06T11:23:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- 172.21.15.7:0/1589586764 wait complete. 2023-10-06T11:23:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 Processor -- start 2023-10-06T11:23:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 -- start start 2023-10-06T11:23:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.423+0000 7fb238b7c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb234068370 0x7fb2340c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340c9950 0x7fb234149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234159c80 con 0x7fb2340ac070 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341581e0 con 0x7fb234068370 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340c5b40 con 0x7fb2340c9950 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54232/0 (socket says 172.21.15.7:54232) 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 -- 172.21.15.7:0/935332878 learned_addr learned my addr 172.21.15.7:0/935332878 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb2337fe700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb234068370 0x7fb2340c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb233fff700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340c9950 0x7fb234149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 -- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340c9950 msgr2=0x7fb234149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340c9950 0x7fb234149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 -- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb234068370 msgr2=0x7fb2340c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb234068370 0x7fb2340c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 -- 172.21.15.7:0/935332878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009420 con 0x7fb2340ac070 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb232ffd700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0091e0 tx=0x7fb21c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00bbe0 con 0x7fb2340ac070 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c011420 con 0x7fb2340ac070 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2340c5e20 con 0x7fb2340ac070 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0117e0 con 0x7fb2340ac070 2023-10-06T11:23:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2341499d0 con 0x7fb2340ac070 2023-10-06T11:23:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb234157ee0 con 0x7fb2340ac070 2023-10-06T11:23:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.427+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb21c011bd0 con 0x7fb2340ac070 2023-10-06T11:23:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.431+0000 7fb22b7fe700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 0x7fb214062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.431+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb21c07e410 con 0x7fb2340ac070 2023-10-06T11:23:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.431+0000 7fb2337fe700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 0x7fb214062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.431+0000 7fb2337fe700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 0x7fb214062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2240076a0 tx=0x7fb2240098f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.439+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb21c04e7a0 con 0x7fb2340ac070 2023-10-06T11:23:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.639+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb23404b5c0 con 0x7fb2340ac070 2023-10-06T11:23:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.639+0000 7fb22b7fe700 1 -- 172.21.15.7:0/935332878 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7fb21c051d80 con 0x7fb2340ac070 2023-10-06T11:23:08.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:08.645 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 msgr2=0x7fb214062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 0x7fb214062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2240076a0 tx=0x7fb2240098f0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 msgr2=0x7fb234159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0091e0 tx=0x7fb21c017000 comp rx=0 tx=0).stop 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 shutdown_connections 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb214060550 0x7fb214062a00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb2340c9950 0x7fb234149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2340ac070 0x7fb234159740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 --2- 172.21.15.7:0/935332878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb234068370 0x7fb2340c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 >> 172.21.15.7:0/935332878 conn(0x7fb23406f380 msgr2=0x7fb2340aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 shutdown_connections 2023-10-06T11:23:08.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:08.643+0000 7fb238b7c700 1 -- 172.21.15.7:0/935332878 wait complete. 2023-10-06T11:23:08.649 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:09.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:23:02.931201+0000', 'modified': '2023-10-06T11:23:06.573482+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5211}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5211': {'gid': 5211, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3888808928', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3888808928}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3888808928}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:23:09.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:23:09.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:23:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.179+0000 7f73b6b90700 1 Processor -- start 2023-10-06T11:23:09.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.179+0000 7f73b6b90700 1 -- start start 2023-10-06T11:23:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.179+0000 7f73b6b90700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f73b00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b015cac0 con 0x7f73b00af0e0 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b015d640 con 0x7f73b0156bf0 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b015dad0 con 0x7f73b00affb0 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73affff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73b00af0e0 0x7f73b01564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49748/0 (socket says 172.21.15.7:49748) 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73affff700 1 -- 172.21.15.7:0/3864082483 learned_addr learned my addr 172.21.15.7:0/3864082483 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f73b00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663894737 0 0) 0x7f73b015cac0 con 0x7f73b00af0e0 2023-10-06T11:23:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f739c003570 con 0x7f73b00af0e0 2023-10-06T11:23:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902901181 0 0) 0x7f73b015d640 con 0x7f73b0156bf0 2023-10-06T11:23:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73b015cac0 con 0x7f73b0156bf0 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1094931616 0 0) 0x7f739c003570 con 0x7f73b00af0e0 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 msgr2=0x7f73b00b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f73b00b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 >> v1:172.21.15.50:6789/0 conn(0x7f73b0156bf0 legacy=0x7f73b006e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00b21c0 con 0x7f73b00af0e0 2023-10-06T11:23:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f7398001770 con 0x7f73b00af0e0 2023-10-06T11:23:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 >> v1:172.21.15.7:6789/0 conn(0x7f73b00af0e0 legacy=0x7f73b01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739c003c20 0x7f739c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c004810 0x7f739c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f739c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c00bbf0 con 0x7f739c003c20 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c00bd30 con 0x7f739c004810 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c00bef0 con 0x7f73b00affb0 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739c003c20 0x7f739c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73affff700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c004810 0x7f739c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b512d700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f739c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 -- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 msgr2=0x7f739c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00affb0 0x7f739c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 -- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c004810 msgr2=0x7f739c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c004810 0x7f739c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 -- 172.21.15.7:0/3864082483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c003570 con 0x7f739c003c20 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b492c700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739c003c20 0x7f739c004010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009c00 tx=0x7f73a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99570033d874921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000e710 con 0x7f739c003c20 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a000e870 con 0x7f739c003c20 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73af7fe700 1 -- 172.21.15.7:0/3864082483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000ec30 con 0x7f739c003c20 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- 172.21.15.7:0/3864082483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739c003c20 msgr2=0x7f739c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 --2- 172.21.15.7:0/3864082483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739c003c20 0x7f739c004010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009c00 tx=0x7f73a00091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b512d700 1 -- 172.21.15.7:0/3864082483 reap_dead start 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- 172.21.15.7:0/3864082483 shutdown_connections 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- 172.21.15.7:0/3864082483 >> 172.21.15.7:0/3864082483 conn(0x7f73b0067e30 msgr2=0x7f73b0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.183+0000 7f73b6b90700 1 -- 172.21.15.7:0/3864082483 shutdown_connections 2023-10-06T11:23:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 -- 172.21.15.7:0/3864082483 wait complete. 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 Processor -- start 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 -- start start 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c003c20 0x7f73b0153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00ab850 0x7f73b00abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01550e0 con 0x7f73b00ae660 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00ac200 con 0x7f739c003c20 2023-10-06T11:23:09.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73b6b90700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01544d0 con 0x7f73b00ab850 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.187+0000 7f73affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54250/0 (socket says 172.21.15.7:54250) 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 -- 172.21.15.7:0/2670912302 learned_addr learned my addr 172.21.15.7:0/2670912302 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73b492c700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c003c20 0x7f73b0153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73b512d700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00ab850 0x7f73b00abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 -- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00ab850 msgr2=0x7f73b00abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00ab850 0x7f73b00abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 -- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c003c20 msgr2=0x7f73b0153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c003c20 0x7f73b0153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 -- 172.21.15.7:0/2670912302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b01547b0 con 0x7f73b00ae660 2023-10-06T11:23:09.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73affff700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7398007a90 tx=0x7f73b00b1390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73acff9700 1 -- 172.21.15.7:0/2670912302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800eb40 con 0x7f73b00ae660 2023-10-06T11:23:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73acff9700 1 -- 172.21.15.7:0/2670912302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739800eca0 con 0x7f73b00ae660 2023-10-06T11:23:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73acff9700 1 -- 172.21.15.7:0/2670912302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398013420 con 0x7f73b00ae660 2023-10-06T11:23:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0154a90 con 0x7f73b00ae660 2023-10-06T11:23:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.191+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b00aead0 con 0x7f73b00ae660 2023-10-06T11:23:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73acff9700 1 -- 172.21.15.7:0/2670912302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7398013790 con 0x7f73b00ae660 2023-10-06T11:23:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73b8e39740 1 Processor -- start 2023-10-06T11:23:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73b8e39740 1 -- start start 2023-10-06T11:23:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73acff9700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 0x7f7390062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73acff9700 1 -- 172.21.15.7:0/2670912302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f739807d0c0 con 0x7f73b00ae660 2023-10-06T11:23:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73b512d700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 0x7f7390062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1466630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.195+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x14670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1394f30 con 0x11e9710 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1390ed0 con 0x11e9ae0 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x13570b0 con 0x1234790 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b492c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1466630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54264/0 (socket says 172.21.15.7:54264) 2023-10-06T11:23:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 -- 172.21.15.7:0/1161583077 learned_addr learned my addr 172.21.15.7:0/1161583077 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b512d700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 0x7f7390062a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c080 tx=0x7f73a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b512d700 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x14670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 -- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 msgr2=0x14670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x14670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 -- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 msgr2=0x1466630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1466630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 -- 172.21.15.7:0/1161583077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12359e0 con 0x11e9710 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b512d700 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x14670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73affff700 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7398050d30 tx=0x7f739807b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13836e5fe5b5a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73a5ffb700 1 -- 172.21.15.7:0/1161583077 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980542c0 con 0x11e9710 2023-10-06T11:23:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73a5ffb700 1 -- 172.21.15.7:0/1161583077 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7398058010 con 0x11e9710 2023-10-06T11:23:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73a5ffb700 1 -- 172.21.15.7:0/1161583077 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739804f170 con 0x11e9710 2023-10-06T11:23:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- 172.21.15.7:0/1161583077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 msgr2=0x1466b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7398050d30 tx=0x7f739807b200 comp rx=0 tx=0).stop 2023-10-06T11:23:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- 172.21.15.7:0/1161583077 shutdown_connections 2023-10-06T11:23:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x14670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x1466b70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1161583077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1466630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- 172.21.15.7:0/1161583077 >> 172.21.15.7:0/1161583077 conn(0x11e9eb0 msgr2=0x1423060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- 172.21.15.7:0/1161583077 shutdown_connections 2023-10-06T11:23:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- 172.21.15.7:0/1161583077 wait complete. 2023-10-06T11:23:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 -- start start 2023-10-06T11:23:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.199+0000 7f73b8e39740 1 Processor -- start 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x146b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x1420c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1356d50 con 0x11e9710 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x132f970 con 0x11e9ae0 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x134a430 con 0x1234790 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 10 client.0 _ll_register_callbacks cb 0x1333b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54274/0 (socket says 172.21.15.7:54274) 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 -- 172.21.15.7:0/4140648959 learned_addr learned my addr 172.21.15.7:0/4140648959 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b512d700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x1420c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73affff700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x146b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 -- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 msgr2=0x1420c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x1420c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 -- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 msgr2=0x146b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x146b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 -- 172.21.15.7:0/4140648959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b00aead0 con 0x11e9710 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b492c700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f739c0057f0 tx=0x7f73a001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71bdae8af1651f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f738effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0022040 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f73a0013680 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.203+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x132fcc0 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a00095b0 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 10 client.5388 fetch_fsmap learned FSMap version 17 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x135f370 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x132fcc0 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f73a0011070 con 0x11e9710 2023-10-06T11:23:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 10 client.5388 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 10 client.5388 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 20 client.5388 populate_metadata read hostname 'smithi007' 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x14223c0 0x1475de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 4 client.5388 mds_command: new command op to 5211 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 --> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x13333e0 con 0x14223c0 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73affff700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x14223c0 0x1475de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73affff700 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x14223c0 0x1475de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711fab73e7e5a7d0 server_cookie=e9025afc8f4143a2 in_seq=0 out_seq=0 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 10 client.5388 ms_handle_connect on v2:172.21.15.7:6834/3888808928 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 1 -- 172.21.15.7:0/4140648959 <== mds.0 v2:172.21.15.7:6834/3888808928 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x13333e0 con 0x14223c0 2023-10-06T11:23:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 10 client.5388 handle_command_reply: tid=0 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 client.5388 shutdown 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 msgr2=0x146ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f739c0057f0 tx=0x7f73a001a000 comp rx=0 tx=0).stop 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 shutdown_connections 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x14223c0 0x1475de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x1234790 0x1420c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x146b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f738effd700 0 client.5388 ms_handle_reset on v2:172.21.15.7:6834/3888808928 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 --2- 172.21.15.7:0/4140648959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x146ac40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 >> 172.21.15.7:0/4140648959 conn(0x11e9eb0 msgr2=0x144a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 shutdown_connections 2023-10-06T11:23:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 1 -- 172.21.15.7:0/4140648959 wait complete. 2023-10-06T11:23:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.207+0000 7f73b8e39740 20 client.5388 trim_cache size 0 max 16384 2023-10-06T11:23:09.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 Processor -- start 2023-10-06T11:23:09.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 -- start start 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9710 0x1469650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1469b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x13945d0 con 0x1234790 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1366010 con 0x11e9ae0 2023-10-06T11:23:09.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1364040 con 0x11e9710 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54284/0 (socket says 172.21.15.7:54284) 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 -- 172.21.15.7:0/1443192275 learned_addr learned my addr 172.21.15.7:0/1443192275 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b492c700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9710 0x1469650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73affff700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1469b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 -- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9710 msgr2=0x1469650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9710 0x1469650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 -- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 msgr2=0x1469b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1469b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 -- 172.21.15.7:0/1443192275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b0157400 con 0x1234790 2023-10-06T11:23:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73affff700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1469b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73b512d700 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73b015d5b0 tx=0x7f73a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaf25573ebe5bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73a67fc700 1 -- 172.21.15.7:0/1443192275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8006a30 con 0x1234790 2023-10-06T11:23:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.227+0000 7f73a67fc700 1 -- 172.21.15.7:0/1443192275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8026070 con 0x1234790 2023-10-06T11:23:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73a67fc700 1 -- 172.21.15.7:0/1443192275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8006da0 con 0x1234790 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/1443192275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 msgr2=0x1445e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73b015d5b0 tx=0x7f73a8017000 comp rx=0 tx=0).stop 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/1443192275 shutdown_connections 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1234790 0x1445e90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x11e9ae0 0x1469b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- 172.21.15.7:0/1443192275 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9710 0x1469650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/1443192275 >> 172.21.15.7:0/1443192275 conn(0x11e9eb0 msgr2=0x1423020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/1443192275 shutdown_connections 2023-10-06T11:23:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/1443192275 wait complete. 2023-10-06T11:23:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- start start 2023-10-06T11:23:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 Processor -- start 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x14c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1234790 0x14452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x13ee7d0 con 0x11e9710 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x1365610 con 0x1234790 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x1364f40 con 0x11e9ae0 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 10 client.0 _ll_register_callbacks cb 0x135c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:23:09.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b492c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x14c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:38330/0 (socket says 172.21.15.7:38330) 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 -- 172.21.15.7:0/922445920 learned_addr learned my addr 172.21.15.7:0/922445920 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b512d700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1234790 0x14452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 -- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1234790 msgr2=0x14452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1234790 0x14452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 -- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 msgr2=0x14c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x14c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 -- 172.21.15.7:0/922445920 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x12359e0 con 0x11e9ae0 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73affff700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7398002040 tx=0x7f7398068040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ead42486702a05a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f738effd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3301/0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739806f070 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mon.2 v2:172.21.15.50:3301/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f73980542c0 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.231+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7398058010 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f739806b040 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 10 client.5000 fetch_fsmap learned FSMap version 17 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x7f73a800f040 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mon.2 v2:172.21.15.50:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7398058010 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mon.2 v2:172.21.15.50:3301/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7398050520 con 0x11e9ae0 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 10 client.5000 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 10 client.5000 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:23:09.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 20 client.5000 populate_metadata read hostname 'smithi007' 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x1445be0 0x1445fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 4 client.5000 mds_command: new command op to 5211 tid=0[{"prefix": "status"}] 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 --> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1365410 con 0x1445be0 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b492c700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x1445be0 0x1445fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 10 client.5000 ms_handle_connect on v2:172.21.15.7:6834/3888808928 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b492c700 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x1445be0 0x1445fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184ae2cb57b40be5 server_cookie=71c6664f69b5fe6e in_seq=0 out_seq=0 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 1 -- 172.21.15.7:0/922445920 <== mds.0 v2:172.21.15.7:6834/3888808928 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x1365410 con 0x1445be0 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 10 client.5000 handle_command_reply: tid=0 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 client.5000 shutdown 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 msgr2=0x14c0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7398002040 tx=0x7f7398068040 comp rx=0 tx=0).stop 2023-10-06T11:23:09.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 shutdown_connections 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:6834/3888808928,v1:172.21.15.7:6835/3888808928] conn(0x1445be0 0x1445fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x1234790 0x14452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x11e9ae0 0x14c0f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 --2- 172.21.15.7:0/922445920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x11e9710 0x14c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 >> 172.21.15.7:0/922445920 conn(0x11e9eb0 msgr2=0x1422790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f738effd700 0 client.5000 ms_handle_reset on v2:172.21.15.7:6834/3888808928 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 shutdown_connections 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 1 -- 172.21.15.7:0/922445920 wait complete. 2023-10-06T11:23:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b8e39740 20 client.5000 trim_cache size 0 max 16384 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "88789558-6143-4e37-9ef9-c1260e585533", 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5211, 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 5.6554827540000003, 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 17, 2023-10-06T11:23:09.245 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 39, 2023-10-06T11:23:09.246 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 39, 2023-10-06T11:23:09.246 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.616543815 2023-10-06T11:23:09.246 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:09.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 msgr2=0x7f7390062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 0x7f7390062a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c080 tx=0x7f73a8009040 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 msgr2=0x7f73b013d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7398007a90 tx=0x7f73b00b1390 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 shutdown_connections 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7390060550 0x7f7390062a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f73b00ab850 0x7f73b00abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b00ae660 0x7f73b013d8a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 --2- 172.21.15.7:0/2670912302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c003c20 0x7f73b0153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 >> 172.21.15.7:0/2670912302 conn(0x7f73b0067e30 msgr2=0x7f73b014ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 shutdown_connections 2023-10-06T11:23:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.235+0000 7f73b6b90700 1 -- 172.21.15.7:0/2670912302 wait complete. 2023-10-06T11:23:09.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:23:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 Processor -- start 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- start start 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26015d0f0 0x7fc260160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc26006de10 0x7fc2600c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2601617c0 con 0x7fc26015d4c0 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc260161aa0 con 0x7fc26006de10 2023-10-06T11:23:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc260161d80 con 0x7fc26015d0f0 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc265367700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc26015d4c0 0x7fc2600cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49760/0 (socket says 172.21.15.7:49760) 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc265367700 1 -- 172.21.15.7:0/4010537119 learned_addr learned my addr 172.21.15.7:0/4010537119 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 --2- 172.21.15.7:0/4010537119 >> v2:172.21.15.50:3300/0 conn(0x7fc26006de10 0x7fc2600c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc264b66700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26015d0f0 0x7fc260160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25f7fe700 1 -- 172.21.15.7:0/4010537119 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939541236 0 0) 0x7fc2601617c0 con 0x7fc26015d4c0 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25f7fe700 1 -- 172.21.15.7:0/4010537119 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003570 con 0x7fc26015d4c0 2023-10-06T11:23:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 -- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26015d0f0 msgr2=0x7fc260160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26015d0f0 0x7fc260160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 -- 172.21.15.7:0/4010537119 >> v1:172.21.15.7:6789/0 conn(0x7fc26015d4c0 legacy=0x7fc2600cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 -- 172.21.15.7:0/4010537119 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600ad220 con 0x7fc26006de10 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25ffff700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26006de10 0x7fc2600c9990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc250009e60 tx=0x7fc250009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e8689d90b89216 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25f7fe700 1 -- 172.21.15.7:0/4010537119 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000e070 con 0x7fc26006de10 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25f7fe700 1 -- 172.21.15.7:0/4010537119 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000bab0 con 0x7fc26006de10 2023-10-06T11:23:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc25f7fe700 1 -- 172.21.15.7:0/4010537119 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000be70 con 0x7fc26006de10 2023-10-06T11:23:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26006de10 msgr2=0x7fc2600c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26006de10 0x7fc2600c9990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc250009e60 tx=0x7fc250009780 comp rx=0 tx=0).stop 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- 172.21.15.7:0/4010537119 shutdown_connections 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26006de10 0x7fc2600c9990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 --2- 172.21.15.7:0/4010537119 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26015d0f0 0x7fc260160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- 172.21.15.7:0/4010537119 >> 172.21.15.7:0/4010537119 conn(0x7fc260066840 msgr2=0x7fc260066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.363+0000 7fc266dca700 1 -- 172.21.15.7:0/4010537119 shutdown_connections 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 -- 172.21.15.7:0/4010537119 wait complete. 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 Processor -- start 2023-10-06T11:23:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 -- start start 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 0x7fc2601565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 0x7fc2600afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b0a50 con 0x7fc26015d4c0 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b04a0 con 0x7fc26015d0f0 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc266dca700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b0770 con 0x7fc26006de10 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc265367700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc25ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 0x7fc2600afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc264b66700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 0x7fc2601565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc25ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 0x7fc2600afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:55240/0 (socket says 172.21.15.7:55240) 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc25ffff700 1 -- 172.21.15.7:0/4101417710 learned_addr learned my addr 172.21.15.7:0/4101417710 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.367+0000 7fc265367700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54300/0 (socket says 172.21.15.7:54300) 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 -- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 msgr2=0x7fc2601565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 0x7fc2601565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 -- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 msgr2=0x7fc2600afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 0x7fc2600afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 -- 172.21.15.7:0/4101417710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc250009a20 con 0x7fc26015d4c0 2023-10-06T11:23:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc264b66700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 0x7fc2601565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc265367700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc2580058d0 tx=0x7fc258007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258005900 con 0x7fc26015d4c0 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258005a60 con 0x7fc26015d4c0 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc260158ab0 con 0x7fc26015d4c0 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258005e00 con 0x7fc26015d4c0 2023-10-06T11:23:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc260159070 con 0x7fc26015d4c0 2023-10-06T11:23:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26004be00 con 0x7fc26015d4c0 2023-10-06T11:23:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc2580074f0 con 0x7fc26015d4c0 2023-10-06T11:23:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 0x7fc240062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.371+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc25807bea0 con 0x7fc26015d4c0 2023-10-06T11:23:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.375+0000 7fc264b66700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 0x7fc240062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.375+0000 7fc264b66700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 0x7fc240062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc248011480 tx=0x7fc2480091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.379+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc25804c1f0 con 0x7fc26015d4c0 2023-10-06T11:23:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.579+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc2600ad500 con 0x7fc26015d4c0 2023-10-06T11:23:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.579+0000 7fc25cff9700 1 -- 172.21.15.7:0/4101417710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc2580077a0 con 0x7fc26015d4c0 2023-10-06T11:23:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:23:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 msgr2=0x7fc240062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 0x7fc240062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc248011480 tx=0x7fc2480091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 msgr2=0x7fc2600aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc2580058d0 tx=0x7fc258007d10 comp rx=0 tx=0).stop 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 shutdown_connections 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc240060550 0x7fc240062a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc26015d4c0 0x7fc2600aff60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26015d0f0 0x7fc2600afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 --2- 172.21.15.7:0/4101417710 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc26006de10 0x7fc2601565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 >> 172.21.15.7:0/4101417710 conn(0x7fc260066840 msgr2=0x7fc260160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 shutdown_connections 2023-10-06T11:23:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:09.583+0000 7fc266dca700 1 -- 172.21.15.7:0/4101417710 wait complete. 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:23:09.604 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:09.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.605 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.611 INFO:teuthology.orchestra.run.smithi007.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42512sec preferred_lft 42512sec 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:c9:9f:83:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.612 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ba:37:aa:5a:16:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::68f6:b5ff:fe13:5c60/64 scope link 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ee:6e:ff:5d:f0:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec6e:ffff:fe5d:f075/64 scope link 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.613 INFO:teuthology.orchestra.run.smithi007.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:23:09.614 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ba:37:aa:5a:16:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:23:09.614 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::b837:aaff:fe5a:1610/64 scope link 2023-10-06T11:23:09.614 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:09.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.614 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:23:09.627 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-06T11:23:09.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.628 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-06T11:23:09.677 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-06T11:23:09.677 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:23:09.677 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:23:09.677 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:23:09.678 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:23:09.678 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:23:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:23:09.678 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:23:09.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.719 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:23:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:23:09.725 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:23:09.726 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:09.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:09.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-06T11:23:09.741 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T11:23:09.741 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:23:09.741 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:09.748 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:09.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:23:09.749 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:09.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:09.812 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:09.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:09.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:09.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:09.828 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:09.834 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:23:09.834 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T11:23:09.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:09.895 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:09.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:09.905 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:09.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:09.905 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:09.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-06T11:23:09.955+0000 7fcdba2a0380 -1 init, newargv = 0x55defb2dfad0 newargc=15 2023-10-06T11:23:09.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43280]: starting ceph client 2023-10-06T11:23:09.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43280]: starting fuse 2023-10-06T11:23:11.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:11.030 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:11.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:11.088 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:11.089 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:11.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:11.090 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:11.097 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:23:11.098 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-06T11:23:11.098 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:23:11.099 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-06T11:23:11.100 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-06T11:23:11.182 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.43280.asok 2023-10-06T11:23:11.368 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43280.asok 2023-10-06T11:23:11.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:11.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43280.asok status 2023-10-06T11:23:11.508 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-06T11:23:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:23:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "43280", 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-06T11:23:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5406, 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5406 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:23:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2534333894 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:23:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2534333894 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5406 192.168.0.1:0/2534333894", 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/2534333894", 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 17, 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 39, 2023-10-06T11:23:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:23:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-06T11:23:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-06T11:23:11.513 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:11.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:11.518 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-06T11:23:11.518 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:11.526 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-06T11:23:11.526 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:11.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:11.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:11.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:23:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.663+0000 7efda0bbc700 1 Processor -- start 2023-10-06T11:23:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.663+0000 7efda0bbc700 1 -- start start 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06eb80 0x7efd9c06bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd9c155850 0x7efd9c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c06e6f0 con 0x7efd9c06bad0 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c1561f0 con 0x7efd9c155850 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c0cbc70 con 0x7efd9c06eb80 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efd9c06bad0 0x7efd9c0cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49764/0 (socket says 172.21.15.7:49764) 2023-10-06T11:23:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9b7fe700 1 -- 172.21.15.7:0/2995526963 learned_addr learned my addr 172.21.15.7:0/2995526963 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2995526963 >> v2:172.21.15.7:3300/0 conn(0x7efd9c155850 0x7efd9c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9affd700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06eb80 0x7efd9c06bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 -- 172.21.15.7:0/2995526963 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06eb80 msgr2=0x7efd9c06bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06eb80 0x7efd9c06bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 -- 172.21.15.7:0/2995526963 >> v1:172.21.15.7:6789/0 conn(0x7efd9c06bad0 legacy=0x7efd9c0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 -- 172.21.15.7:0/2995526963 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0cbf50 con 0x7efd9c155850 2023-10-06T11:23:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c155cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efd94009e60 tx=0x7efd94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879cedb58102ed9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9a7fc700 1 -- 172.21.15.7:0/2995526963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9400e070 con 0x7efd9c155850 2023-10-06T11:23:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9a7fc700 1 -- 172.21.15.7:0/2995526963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9400bb60 con 0x7efd9c155850 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efd9a7fc700 1 -- 172.21.15.7:0/2995526963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94012430 con 0x7efd9c155850 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 -- 172.21.15.7:0/2995526963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 msgr2=0x7efd9c155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.667+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c155cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efd94009e60 tx=0x7efd94009780 comp rx=0 tx=0).stop 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- 172.21.15.7:0/2995526963 shutdown_connections 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c155cb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2995526963 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06eb80 0x7efd9c06bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- 172.21.15.7:0/2995526963 >> 172.21.15.7:0/2995526963 conn(0x7efd9c0cc190 msgr2=0x7efd9c0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- 172.21.15.7:0/2995526963 shutdown_connections 2023-10-06T11:23:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- 172.21.15.7:0/2995526963 wait complete. 2023-10-06T11:23:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 Processor -- start 2023-10-06T11:23:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- start start 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06bad0 0x7efd9c0c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c06eb80 0x7efd9c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c0c3780 con 0x7efd9c155850 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c16c870 con 0x7efd9c06eb80 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efda0bbc700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c16cb50 con 0x7efd9c06bad0 2023-10-06T11:23:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54326/0 (socket says 172.21.15.7:54326) 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 -- 172.21.15.7:0/2049258855 learned_addr learned my addr 172.21.15.7:0/2049258855 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9affd700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c06eb80 0x7efd9c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9b7fe700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06bad0 0x7efd9c0c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 -- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06bad0 msgr2=0x7efd9c0c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06bad0 0x7efd9c0c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 -- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c06eb80 msgr2=0x7efd9c144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c06eb80 0x7efd9c144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.671+0000 7efd9bfff700 1 -- 172.21.15.7:0/2049258855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94009a20 con 0x7efd9c155850 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd9bfff700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06ae30 tx=0x7efd94019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94027470 con 0x7efd9c155850 2023-10-06T11:23:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94027610 con 0x7efd9c155850 2023-10-06T11:23:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c16ce30 con 0x7efd9c155850 2023-10-06T11:23:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940279d0 con 0x7efd9c155850 2023-10-06T11:23:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c16d3f0 con 0x7efd9c155850 2023-10-06T11:23:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c06d240 con 0x7efd9c155850 2023-10-06T11:23:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efd94024070 con 0x7efd9c155850 2023-10-06T11:23:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.675+0000 7efd837fe700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 0x7efd88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.679+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd9407f4d0 con 0x7efd9c155850 2023-10-06T11:23:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.679+0000 7efd9b7fe700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 0x7efd88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.679+0000 7efd9b7fe700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 0x7efd88062a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.683+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efd9404f820 con 0x7efd9c155850 2023-10-06T11:23:11.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.867+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efd9c16d5e0 con 0x7efd88060550 2023-10-06T11:23:11.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.871+0000 7efd837fe700 1 -- 172.21.15.7:0/2049258855 <== mgr.4102 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7efd8c001ca0 con 0x7efd88060550 2023-10-06T11:23:11.875 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-06T11:23:11.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-06T11:23:11.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs" 2023-10-06T11:23:11.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:23:11.876 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 msgr2=0x7efd88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 0x7efd88062a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c009040 comp rx=0 tx=0).stop 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 msgr2=0x7efd9c150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06ae30 tx=0x7efd94019000 comp rx=0 tx=0).stop 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 shutdown_connections 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efd88060550 0x7efd88062a00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd9c155850 0x7efd9c150cc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c06eb80 0x7efd9c144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 --2- 172.21.15.7:0/2049258855 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efd9c06bad0 0x7efd9c0c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 >> 172.21.15.7:0/2049258855 conn(0x7efd9c0cc190 msgr2=0x7efd9c0c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 shutdown_connections 2023-10-06T11:23:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.875+0000 7efda0bbc700 1 -- 172.21.15.7:0/2049258855 wait complete. 2023-10-06T11:23:11.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:23:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.995+0000 7f2e8a746700 1 Processor -- start 2023-10-06T11:23:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.995+0000 7f2e8a746700 1 -- start start 2023-10-06T11:23:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e84159eb0 0x7f2e840cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8406e770 con 0x7f2e840cefe0 2023-10-06T11:23:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e840ceb50 con 0x7f2e8406e0e0 2023-10-06T11:23:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8406e4f0 con 0x7f2e84159eb0 2023-10-06T11:23:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e837fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e840cefe0 0x7f2e8406d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50210/0 (socket says 172.21.15.7:50210) 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e837fe700 1 -- 172.21.15.7:0/2310097705 learned_addr learned my addr 172.21.15.7:0/2310097705 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2310097705 >> v2:172.21.15.50:3300/0 conn(0x7f2e84159eb0 0x7f2e840cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843996535 0 0) 0x7f2e8406e770 con 0x7f2e840cefe0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e6c003570 con 0x7f2e840cefe0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183320016 0 0) 0x7f2e6c003570 con 0x7f2e840cefe0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 >> v2:172.21.15.50:3300/0 conn(0x7f2e84159eb0 msgr2=0x7f2e840cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 --2- 172.21.15.7:0/2310097705 >> v2:172.21.15.50:3300/0 conn(0x7f2e84159eb0 0x7f2e840cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 >> v1:172.21.15.50:6789/0 conn(0x7f2e8406e0e0 legacy=0x7f2e840ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840ce220 con 0x7f2e840cefe0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f2e78001470 con 0x7f2e840cefe0 2023-10-06T11:23:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 >> v1:172.21.15.7:6789/0 conn(0x7f2e840cefe0 legacy=0x7f2e8406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c003a60 0x7f2e6c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e6c004810 0x7f2e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8406e4f0 con 0x7f2e6c0051c0 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c00c050 con 0x7f2e6c004810 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c00c210 con 0x7f2e6c003a60 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c003a60 0x7f2e6c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e837fe700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e6c004810 0x7f2e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c003a60 msgr2=0x7f2e6c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c003a60 0x7f2e6c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e6c004810 msgr2=0x7f2e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e6c004810 0x7f2e6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2310097705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c003570 con 0x7f2e6c0051c0 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e837fe700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e6c004810 0x7f2e6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2310097705 reap_dead start 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00ba60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0020a0 tx=0x7f2e7c0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc6cfd897d4e265 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0019e0 con 0x7f2e6c0051c0 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c00c8b0 con 0x7f2e6c0051c0 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e82ffd700 1 -- 172.21.15.7:0/2310097705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00cc50 con 0x7f2e6c0051c0 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 -- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 msgr2=0x7f2e6c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:11.999+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00ba60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0020a0 tx=0x7f2e7c0016d0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 -- 172.21.15.7:0/2310097705 shutdown_connections 2023-10-06T11:23:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2310097705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00ba60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 -- 172.21.15.7:0/2310097705 >> 172.21.15.7:0/2310097705 conn(0x7f2e84067e30 msgr2=0x7f2e84068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 -- 172.21.15.7:0/2310097705 shutdown_connections 2023-10-06T11:23:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 -- 172.21.15.7:0/2310097705 wait complete. 2023-10-06T11:23:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 Processor -- start 2023-10-06T11:23:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 -- start start 2023-10-06T11:23:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.003+0000 7f2e8a746700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 0x7f2e84151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840b2ba0 0x7f2e840b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8415bdb0 con 0x7f2e8415b400 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840b35f0 con 0x7f2e840b2ba0 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840b38d0 con 0x7f2e6c0051c0 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37396/0 (socket says 172.21.15.7:37396) 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2383829926 learned_addr learned my addr 172.21.15.7:0/2383829926 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 0x7f2e84151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e837fe700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840b2ba0 0x7f2e840b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 msgr2=0x7f2e84151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 0x7f2e84151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840b2ba0 msgr2=0x7f2e840b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840b2ba0 0x7f2e840b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 -- 172.21.15.7:0/2383829926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840b3bb0 con 0x7f2e8415b400 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 0x7f2e84151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e88ce3700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e840cf3b0 tx=0x7f2e7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0019e0 con 0x7f2e8415b400 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c0078e0 con 0x7f2e8415b400 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e840b3e90 con 0x7f2e8415b400 2023-10-06T11:23:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c007e70 con 0x7f2e8415b400 2023-10-06T11:23:12.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e840b12c0 con 0x7f2e8415b400 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8404be00 con 0x7f2e8415b400 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.007+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2e7c01e8b0 con 0x7f2e8415b400 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.011+0000 7f2e6bfff700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 0x7f2e70062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.011+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2e7c07eaa0 con 0x7f2e8415b400 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.011+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 0x7f2e70062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.011+0000 7f2e83fff700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 0x7f2e70062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006f00 tx=0x7f2e74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.019+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2e7c04edf0 con 0x7f2e8415b400 2023-10-06T11:23:12.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.223+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2e840b1e20 con 0x7f2e8415b400 2023-10-06T11:23:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.911+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7f2e7c04f1c0 con 0x7f2e8415b400 2023-10-06T11:23:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.911+0000 7f2e6bfff700 1 -- 172.21.15.7:0/2383829926 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2e7c052880 con 0x7f2e8415b400 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 msgr2=0x7f2e70062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 0x7f2e70062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006f00 tx=0x7f2e74008040 comp rx=0 tx=0).stop 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 msgr2=0x7f2e8415b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e840cf3b0 tx=0x7f2e7c016000 comp rx=0 tx=0).stop 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 shutdown_connections 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e70060550 0x7f2e70062a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8415b400 0x7f2e8415b870 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840b2ba0 0x7f2e840b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 --2- 172.21.15.7:0/2383829926 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e6c0051c0 0x7f2e84151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 >> 172.21.15.7:0/2383829926 conn(0x7f2e84067e30 msgr2=0x7f2e84158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 shutdown_connections 2023-10-06T11:23:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:12.915+0000 7f2e8a746700 1 -- 172.21.15.7:0/2383829926 wait complete. 2023-10-06T11:23:12.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:23:13.032 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.027+0000 7ff9257b6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:23:13.033 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.026+0000 7f9f963fa000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:23:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.039+0000 7f673507f700 1 Processor -- start 2023-10-06T11:23:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.039+0000 7f673507f700 1 -- start start 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673006a4f0 0x7f67300c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67300c7080 con 0x7f673006b8a0 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67300c7b60 con 0x7f673006a4f0 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67301581c0 con 0x7f67300668b0 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673006a4f0 0x7f67300c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673006a4f0 0x7f67300c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37420/0 (socket says 172.21.15.7:37420) 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 -- 172.21.15.7:0/4180374638 learned_addr learned my addr 172.21.15.7:0/4180374638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672f59e700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 -- 172.21.15.7:0/4180374638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 msgr2=0x7f673006bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673006bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 -- 172.21.15.7:0/4180374638 >> v1:172.21.15.50:6789/0 conn(0x7f673006b8a0 legacy=0x7f6730069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 -- 172.21.15.7:0/4180374638 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67301575b0 con 0x7f673006a4f0 2023-10-06T11:23:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672e59c700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006a4f0 0x7f67300c6b40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f67240099b0 tx=0x7f6724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f7152871448f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672dd9b700 1 -- 172.21.15.7:0/4180374638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724009ea0 con 0x7f673006a4f0 2023-10-06T11:23:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672dd9b700 1 -- 172.21.15.7:0/4180374638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400b860 con 0x7f673006a4f0 2023-10-06T11:23:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f672dd9b700 1 -- 172.21.15.7:0/4180374638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400bc00 con 0x7f673006a4f0 2023-10-06T11:23:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- 172.21.15.7:0/4180374638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006a4f0 msgr2=0x7f67300c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006a4f0 0x7f67300c6b40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f67240099b0 tx=0x7f6724009250 comp rx=0 tx=0).stop 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- 172.21.15.7:0/4180374638 shutdown_connections 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673006bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 --2- 172.21.15.7:0/4180374638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006a4f0 0x7f67300c6b40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- 172.21.15.7:0/4180374638 >> 172.21.15.7:0/4180374638 conn(0x7f67300cc2f0 msgr2=0x7f67300cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- 172.21.15.7:0/4180374638 shutdown_connections 2023-10-06T11:23:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.043+0000 7f673507f700 1 -- 172.21.15.7:0/4180374638 wait complete. 2023-10-06T11:23:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 Processor -- start 2023-10-06T11:23:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- start start 2023-10-06T11:23:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f673006a4f0 0x7f6730156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730151ab0 con 0x7f673006b8a0 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673014a7e0 con 0x7f673006a4f0 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6730157110 con 0x7f67300668b0 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37422/0 (socket says 172.21.15.7:37422) 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 -- 172.21.15.7:0/4097088623 learned_addr learned my addr 172.21.15.7:0/4097088623 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672e59c700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f673006a4f0 0x7f6730156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672ed9d700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 -- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 msgr2=0x7f673014a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673014a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 -- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f673006a4f0 msgr2=0x7f6730156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f673006a4f0 0x7f6730156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 -- 172.21.15.7:0/4097088623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240094f0 con 0x7f673006b8a0 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672ed9d700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673014a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f672f59e700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6728009f20 tx=0x7f672800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800cb70 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672800f040 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6730158e60 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728014480 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.047+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6730158810 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7f6728014830 con 0x7f673006b8a0 2023-10-06T11:23:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6730158ff0 con 0x7f673006b8a0 2023-10-06T11:23:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f6716ffd700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 0x7f671c062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f672ed9d700 1 -- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 msgr2=0x7f671c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f672807d770 con 0x7f673006b8a0 2023-10-06T11:23:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.051+0000 7f672ed9d700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 0x7f671c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.059+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f672804dac0 con 0x7f673006b8a0 2023-10-06T11:23:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.223+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f673004be00 con 0x7f673006b8a0 2023-10-06T11:23:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.227+0000 7f6716ffd700 1 -- 172.21.15.7:0/4097088623 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f67280510e0 con 0x7f673006b8a0 2023-10-06T11:23:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 msgr2=0x7f671c062a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 0x7f671c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 msgr2=0x7f673014eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6728009f20 tx=0x7f672800b040 comp rx=0 tx=0).stop 2023-10-06T11:23:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 shutdown_connections 2023-10-06T11:23:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f671c0605a0 0x7f671c062a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673006b8a0 0x7f673014eee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f673006a4f0 0x7f6730156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 --2- 172.21.15.7:0/4097088623 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f67300668b0 0x7f673014a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 >> 172.21.15.7:0/4097088623 conn(0x7f67300cc2f0 msgr2=0x7f6730150360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 shutdown_connections 2023-10-06T11:23:13.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.231+0000 7f673507f700 1 -- 172.21.15.7:0/4097088623 wait complete. 2023-10-06T11:23:13.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:23:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.351+0000 7fa59f073700 1 Processor -- start 2023-10-06T11:23:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.351+0000 7fa59f073700 1 -- start start 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5981554c0 0x7fa598069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa59806a210 0x7fa5980c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5980c7af0 con 0x7fa5981550f0 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59806a650 con 0x7fa59806a210 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980cb3e0 con 0x7fa5981554c0 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e071700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5981550f0 0x7fa5980ca8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50224/0 (socket says 172.21.15.7:50224) 2023-10-06T11:23:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e071700 1 -- 172.21.15.7:0/3101785269 learned_addr learned my addr 172.21.15.7:0/3101785269 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 --2- 172.21.15.7:0/3101785269 >> v2:172.21.15.7:3300/0 conn(0x7fa59806a210 0x7fa5980c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59d870700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5981554c0 0x7fa598069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 -- 172.21.15.7:0/3101785269 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5981554c0 msgr2=0x7fa598069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5981554c0 0x7fa598069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59d06f700 1 -- 172.21.15.7:0/3101785269 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724170691 0 0) 0x7fa5980c7af0 con 0x7fa5981550f0 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 -- 172.21.15.7:0/3101785269 >> v1:172.21.15.7:6789/0 conn(0x7fa5981550f0 legacy=0x7fa5980ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 -- 172.21.15.7:0/3101785269 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980cb6c0 con 0x7fa59806a210 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59e872700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa59806a210 0x7fa5980c7f80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa59000af70 tx=0x7fa59000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a105b403f6c685de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59d06f700 1 -- 172.21.15.7:0/3101785269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000f070 con 0x7fa59806a210 2023-10-06T11:23:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59d06f700 1 -- 172.21.15.7:0/3101785269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59000a2e0 con 0x7fa59806a210 2023-10-06T11:23:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59d06f700 1 -- 172.21.15.7:0/3101785269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000cc70 con 0x7fa59806a210 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- 172.21.15.7:0/3101785269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa59806a210 msgr2=0x7fa5980c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa59806a210 0x7fa5980c7f80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa59000af70 tx=0x7fa59000a890 comp rx=0 tx=0).stop 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- 172.21.15.7:0/3101785269 shutdown_connections 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa59806a210 0x7fa5980c7f80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 --2- 172.21.15.7:0/3101785269 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa5981554c0 0x7fa598069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- 172.21.15.7:0/3101785269 >> 172.21.15.7:0/3101785269 conn(0x7fa598067e30 msgr2=0x7fa598068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- 172.21.15.7:0/3101785269 shutdown_connections 2023-10-06T11:23:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.355+0000 7fa59f073700 1 -- 172.21.15.7:0/3101785269 wait complete. 2023-10-06T11:23:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 Processor -- start 2023-10-06T11:23:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- start start 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa59806a210 0x7fa59814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5981550f0 0x7fa598169470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5981524e0 con 0x7fa598151b30 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59814f1a0 con 0x7fa5981550f0 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5981683d0 con 0x7fa59806a210 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37444/0 (socket says 172.21.15.7:37444) 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 -- 172.21.15.7:0/3162959212 learned_addr learned my addr 172.21.15.7:0/3162959212 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59d870700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5981550f0 0x7fa598169470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e071700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa59806a210 0x7fa59814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 -- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa59806a210 msgr2=0x7fa59814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa59806a210 0x7fa59814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 -- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5981550f0 msgr2=0x7fa598169470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5981550f0 0x7fa598169470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 -- 172.21.15.7:0/3162959212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59000ab30 con 0x7fa598151b30 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59e872700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa598069a20 tx=0x7fa590019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590003310 con 0x7fa598151b30 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa590009cc0 con 0x7fa598151b30 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5981686b0 con 0x7fa598151b30 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590027660 con 0x7fa598151b30 2023-10-06T11:23:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.359+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa598168c70 con 0x7fa598151b30 2023-10-06T11:23:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59806cf60 con 0x7fa598151b30 2023-10-06T11:23:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7fa59001d410 con 0x7fa598151b30 2023-10-06T11:23:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa58e7fc700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 0x7fa57c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa59007e880 con 0x7fa598151b30 2023-10-06T11:23:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa59e071700 1 -- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 msgr2=0x7fa57c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.363+0000 7fa59e071700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 0x7fa57c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.371+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa59004ec50 con 0x7fa598151b30 2023-10-06T11:23:13.470 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.462+0000 7f9f963fa000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:23:13.480 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.475+0000 7ff9257b6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:23:13.520 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.514+0000 7f9f963fa000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:23:13.531 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.527+0000 7ff9257b6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:23:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.527+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa5980c7780 con 0x7fa598151b30 2023-10-06T11:23:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa58e7fc700 1 -- 172.21.15.7:0/3162959212 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa590052270 con 0x7fa598151b30 2023-10-06T11:23:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 msgr2=0x7fa57c062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 0x7fa57c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 msgr2=0x7fa598151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa598069a20 tx=0x7fa590019000 comp rx=0 tx=0).stop 2023-10-06T11:23:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 shutdown_connections 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa57c060670 0x7fa57c062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa598151b30 0x7fa598151fa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5981550f0 0x7fa598169470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 --2- 172.21.15.7:0/3162959212 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa59806a210 0x7fa59814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 >> 172.21.15.7:0/3162959212 conn(0x7fa598067e30 msgr2=0x7fa5980b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 shutdown_connections 2023-10-06T11:23:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.535+0000 7fa59f073700 1 -- 172.21.15.7:0/3162959212 wait complete. 2023-10-06T11:23:13.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:13.552 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:23:13.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T11:23:13.630 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.626+0000 7f9f963fa000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:13.668 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.663+0000 7ff9257b6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:13.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 Processor -- start 2023-10-06T11:23:13.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 -- start start 2023-10-06T11:23:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7d38155e40 0x7f7d38068f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d38156210 con 0x7f7d38069480 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d380c7c40 con 0x7f7d380c8b40 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d37fff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d380699c0 con 0x7f7d38155e40 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d367fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7d38069480 0x7f7d380c80d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50226/0 (socket says 172.21.15.7:50226) 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d367fc700 1 -- 172.21.15.7:0/1947055105 learned_addr learned my addr 172.21.15.7:0/1947055105 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1947055105 >> v2:172.21.15.50:3300/0 conn(0x7f7d38155e40 0x7f7d38068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545736356 0 0) 0x7f7d38156210 con 0x7f7d38069480 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d20003570 con 0x7f7d38069480 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997705704 0 0) 0x7f7d20003570 con 0x7f7d38069480 2023-10-06T11:23:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1947055105 >> v1:172.21.15.50:6789/0 conn(0x7f7d380c8b40 legacy=0x7f7d380adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1947055105 >> v1:172.21.15.7:6789/0 conn(0x7f7d38069480 legacy=0x7f7d380c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1947055105 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380c7310 con 0x7f7d38155e40 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1947055105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d38068f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098e0 tx=0x7f7d280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad15ecd44089b003 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c001470 con 0x7f7d38155e40 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28011420 con 0x7f7d38155e40 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.679+0000 7f7d35ffb700 1 -- 172.21.15.7:0/1947055105 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280117c0 con 0x7f7d38155e40 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- 172.21.15.7:0/1947055105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 msgr2=0x7f7d38068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1947055105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d38068f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098e0 tx=0x7f7d280091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- 172.21.15.7:0/1947055105 shutdown_connections 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1947055105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d38068f40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- 172.21.15.7:0/1947055105 >> 172.21.15.7:0/1947055105 conn(0x7f7d3806ea80 msgr2=0x7f7d3806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- 172.21.15.7:0/1947055105 shutdown_connections 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- 172.21.15.7:0/1947055105 wait complete. 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 Processor -- start 2023-10-06T11:23:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- start start 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7d38069480 0x7f7d380687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d380b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c6370 con 0x7f7d380c8b40 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c5760 con 0x7f7d38155e40 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d37fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c5a40 con 0x7f7d38069480 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d36ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7d38069480 0x7f7d380687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37448/0 (socket says 172.21.15.7:37448) 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d367fc700 1 -- 172.21.15.7:0/1397454392 learned_addr learned my addr 172.21.15.7:0/1397454392 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.683+0000 7f7d377fe700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d380b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7d38069480 msgr2=0x7f7d380687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7d38069480 0x7f7d380687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 msgr2=0x7f7d380b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d380b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28009420 con 0x7f7d380c8b40 2023-10-06T11:23:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d367fc700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c930 tx=0x7f7d2c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c005b30 con 0x7f7d380c8b40 2023-10-06T11:23:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c005c90 con 0x7f7d380c8b40 2023-10-06T11:23:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d380c5d20 con 0x7f7d380c8b40 2023-10-06T11:23:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c007340 con 0x7f7d380c8b40 2023-10-06T11:23:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d38158e50 con 0x7f7d380c8b40 2023-10-06T11:23:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d380c6520 con 0x7f7d380c8b40 2023-10-06T11:23:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7f7d2c012400 con 0x7f7d380c8b40 2023-10-06T11:23:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7d2c07bf20 con 0x7f7d380c8b40 2023-10-06T11:23:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.687+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.695+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d2c049be0 con 0x7f7d380c8b40 2023-10-06T11:23:13.832 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.826+0000 7f9f963fa000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:23:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.871+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d38068d20 con 0x7f7d240605a0 2023-10-06T11:23:13.880 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.875+0000 7ff9257b6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:23:13.882 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:13.878+0000 7f9f963fa000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:13.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.891+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:13.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:13.891+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:23:13.933 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:13.927+0000 7ff9257b6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:14.040 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.034+0000 7f9f963fa000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:23:14.099 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.094+0000 7f9f963fa000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:23:14.101 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.095+0000 7ff9257b6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:23:14.160 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.154+0000 7f9f963fa000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:23:14.163 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.159+0000 7ff9257b6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:23:14.228 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.223+0000 7ff9257b6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:23:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:14.295+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:14.295+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:23:14.473 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.466+0000 7f9f963fa000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:23:14.560 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.555+0000 7ff9257b6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:23:14.590 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.586+0000 7f9f963fa000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:23:14.682 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.679+0000 7ff9257b6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:23:14.906 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:14.902+0000 7f9f963fa000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:23:14.996 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:14.991+0000 7ff9257b6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:23:15.077 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.070+0000 7f9f963fa000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:15.099+0000 7f7d36ffd700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:15.099+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:23:15.144 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.138+0000 7f9f963fa000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:23:15.163 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.159+0000 7ff9257b6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:15.197 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.190+0000 7f9f963fa000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:23:15.229 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.227+0000 7ff9257b6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:23:15.253 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.246+0000 7f9f963fa000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:23:15.283 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.279+0000 7ff9257b6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:23:15.339 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.335+0000 7ff9257b6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:23:15.429 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.422+0000 7f9f963fa000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:15.486 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.482+0000 7f9f963fa000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:23:15.515 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.511+0000 7ff9257b6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:15.550 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.542+0000 7f9f963fa000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:23:15.570 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.567+0000 7ff9257b6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:23:15.634 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.631+0000 7ff9257b6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:23:15.705 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.698+0000 7f9f963fa000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:23:15.766 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:15.762+0000 7f9f963fa000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:23:15.789 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.787+0000 7ff9257b6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:23:15.851 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:15.847+0000 7ff9257b6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:23:16.379 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.374+0000 7f9f963fa000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:23:16.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:16.411+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) v1 ==== 79687+0+0 (secure 0 0 0) 0x7f7d2c04c770 con 0x7f7d380c8b40 2023-10-06T11:23:16.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:16.411+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:16.411+0000 7f7d1effd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:16.474 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:16.471+0000 7ff9257b6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:23:16.573 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.566+0000 7f9f565a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.573 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.566+0000 7f9f565a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.573 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.566+0000 7f9f565a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.573 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.566+0000 7f9f565a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.573 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.566+0000 7f9f565a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.574 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.570+0000 7f9f58da9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.574 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.570+0000 7f9f58da9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.574 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.570+0000 7f9f58da9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.574 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.570+0000 7f9f58da9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:16.574 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:16.570+0000 7f9f58da9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:17.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.419+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 8) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f7d2c0571e0 con 0x7f7d380c8b40 2023-10-06T11:23:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.419+0000 7f7d1effd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.419+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d24069070 con 0x7f7d240605a0 2023-10-06T11:23:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.423+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.423+0000 7f7d36ffd700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d280091e0 tx=0x7f7d28016000 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.543+0000 7f7d1effd700 1 -- 172.21.15.7:0/1397454392 <== mgr.5436 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d24069070 con 0x7f7d240605a0 2023-10-06T11:23:17.548 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule set for path snap_test_dir1 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 msgr2=0x7f7d24062a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d280091e0 tx=0x7f7d28016000 comp rx=0 tx=0).stop 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 msgr2=0x7f7d38154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c930 tx=0x7f7d2c001970 comp rx=0 tx=0).stop 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 shutdown_connections 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7d240605a0 0x7f7d24062a50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38155e40 0x7f7d380b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d380c8b40 0x7f7d38154fa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 --2- 172.21.15.7:0/1397454392 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7d38069480 0x7f7d380687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 >> 172.21.15.7:0/1397454392 conn(0x7f7d3806ea80 msgr2=0x7f7d380c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 shutdown_connections 2023-10-06T11:23:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.547+0000 7f7d37fff700 1 -- 172.21.15.7:0/1397454392 wait complete. 2023-10-06T11:23:17.568 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T11:23:17.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-06T11:23:17.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 Processor -- start 2023-10-06T11:23:17.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 -- start start 2023-10-06T11:23:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 0x7fb8a80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a815d320 con 0x7fb8a80b21d0 2023-10-06T11:23:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a80ce940 con 0x7fb8a80ccdb0 2023-10-06T11:23:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a815c9f0 con 0x7fb8a8159f20 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8a80b21d0 0x7fb8a80cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50230/0 (socket says 172.21.15.7:50230) 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a659c700 1 -- 172.21.15.7:0/2308906548 learned_addr learned my addr 172.21.15.7:0/2308906548 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a5d9b700 1 --2- 172.21.15.7:0/2308906548 >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 0x7fb8a80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.683+0000 7fb8a6d9d700 1 --2- 172.21.15.7:0/2308906548 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162928237 0 0) 0x7fb8a815d320 con 0x7fb8a80b21d0 2023-10-06T11:23:17.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb890003570 con 0x7fb8a80b21d0 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a6d9d700 1 -- 172.21.15.7:0/2308906548 >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 msgr2=0x7fb8a80ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a6d9d700 1 --2- 172.21.15.7:0/2308906548 >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 0x7fb8a80ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a6d9d700 1 -- 172.21.15.7:0/2308906548 >> v1:172.21.15.7:6789/0 conn(0x7fb8a80b21d0 legacy=0x7fb8a80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799078642 0 0) 0x7fb890003570 con 0x7fb8a80b21d0 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a6d9d700 1 -- 172.21.15.7:0/2308906548 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a815ccd0 con 0x7fb8a8159f20 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a5d9b700 1 --2- 172.21.15.7:0/2308906548 >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 0x7fb8a80ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:17.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a6d9d700 1 --2- 172.21.15.7:0/2308906548 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80ca600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e60 tx=0x7fb8a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7cdd36414c4b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000e070 con 0x7fb8a8159f20 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000b7a0 con 0x7fb8a8159f20 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a559a700 1 -- 172.21.15.7:0/2308906548 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c001470 con 0x7fb8a8159f20 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 -- 172.21.15.7:0/2308906548 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 msgr2=0x7fb8a80ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 --2- 172.21.15.7:0/2308906548 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80ca600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009e60 tx=0x7fb8a0009780 comp rx=0 tx=0).stop 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 -- 172.21.15.7:0/2308906548 shutdown_connections 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 --2- 172.21.15.7:0/2308906548 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80ca600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 --2- 172.21.15.7:0/2308906548 >> v2:172.21.15.50:3300/0 conn(0x7fb8a80ccdb0 0x7fb8a80ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 -- 172.21.15.7:0/2308906548 >> 172.21.15.7:0/2308906548 conn(0x7fb8a8067e30 msgr2=0x7fb8a8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:17.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 -- 172.21.15.7:0/2308906548 shutdown_connections 2023-10-06T11:23:17.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.687+0000 7fb8a759e700 1 -- 172.21.15.7:0/2308906548 wait complete. 2023-10-06T11:23:17.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 Processor -- start 2023-10-06T11:23:17.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- start start 2023-10-06T11:23:17.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8a80ccdb0 0x7fb8a80c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80b2a40 con 0x7fb8a80b21d0 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80b2cb0 con 0x7fb8a80ccdb0 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c80b0 con 0x7fb8a8159f20 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37460/0 (socket says 172.21.15.7:37460) 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 -- 172.21.15.7:0/1162405932 learned_addr learned my addr 172.21.15.7:0/1162405932 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a5d9b700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8a80ccdb0 0x7fb8a80c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a6d9d700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 -- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 msgr2=0x7fb8a80c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 -- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8a80ccdb0 msgr2=0x7fb8a80c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8a80ccdb0 0x7fb8a80c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 -- 172.21.15.7:0/1162405932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898009420 con 0x7fb8a80b21d0 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a659c700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c7f0 tx=0x7fb89c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c007360 con 0x7fb8a80b21d0 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c0074c0 con 0x7fb8a80b21d0 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a0009a20 con 0x7fb8a80b21d0 2023-10-06T11:23:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c004830 con 0x7fb8a80b21d0 2023-10-06T11:23:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.691+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a80c8670 con 0x7fb8a80b21d0 2023-10-06T11:23:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.695+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a806cc90 con 0x7fb8a80b21d0 2023-10-06T11:23:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.695+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fb89c004c20 con 0x7fb8a80b21d0 2023-10-06T11:23:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.695+0000 7fb8967fc700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 0x7fb8880628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.695+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fb89c07ba60 con 0x7fb8a80b21d0 2023-10-06T11:23:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.699+0000 7fb8a5d9b700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 0x7fb8880628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.699+0000 7fb8a5d9b700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 0x7fb8880628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80c5030 tx=0x7fb8980091a0 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:17.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.703+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb89c04bfe0 con 0x7fb8a80b21d0 2023-10-06T11:23:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.879+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a806e880 con 0x7fb8880603f0 2023-10-06T11:23:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:17.999+0000 7fb8967fc700 1 -- 172.21.15.7:0/1162405932 <== mgr.5436 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fb8a806e880 con 0x7fb8880603f0 2023-10-06T11:23:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:Retention added to path snap_test_dir1 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 msgr2=0x7fb8880628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 0x7fb8880628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80c5030 tx=0x7fb8980091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 msgr2=0x7fb8a8155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c7f0 tx=0x7fb89c0019e0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 shutdown_connections 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb8880603f0 0x7fb8880628a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb8a8159f20 0x7fb8a80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8a80ccdb0 0x7fb8a80c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 --2- 172.21.15.7:0/1162405932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8a80b21d0 0x7fb8a8155920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 >> 172.21.15.7:0/1162405932 conn(0x7fb8a8067e30 msgr2=0x7fb8a8159460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 shutdown_connections 2023-10-06T11:23:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:18.003+0000 7fb8a759e700 1 -- 172.21.15.7:0/1162405932 wait complete. 2023-10-06T11:23:18.024 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-06T11:23:18.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:23:18.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:23:18.061 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2023-10-06T11:23:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2023-10-06T11:23:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:23:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2023-10-06T11:23:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:23:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2023-10-06T11:23:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.50,[v2:172.21.15.50:3301,v1:172.21.15.50:6790]" 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.065 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2023-10-06T11:23:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2023-10-06T11:23:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2023-10-06T11:23:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-06T11:23:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2023-10-06T11:23:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: client snapdir = .snap 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2023-10-06T11:23:18.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2023-10-06T11:23:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2023-10-06T11:23:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2023-10-06T11:23:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2023-10-06T11:23:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-06T11:23:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-06T11:23:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.076 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:23:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "172.21.15.7,172.21.15.50,[v2:172.21.15.50:3301,v1:172.21.15.50:6790]" 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 88789558-6143-4e37-9ef9-c1260e585533 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.080 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2023-10-06T11:23:18.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:23:18.082 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 20 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 20 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 10 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2023-10-06T11:23:18.083 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.084 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.085 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon op complaint time = 120 2023-10-06T11:23:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: client mount timeout = 600 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: client snapdir = .snap 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = "" 2023-10-06T11:23:18.087 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mds] 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds = 20 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug frag = True 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug scatterstat = True 2023-10-06T11:23:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: mds op complaint time = 180 2023-10-06T11:23:18.089 INFO:teuthology.orchestra.run.smithi050.stdout: mds verify scatter = True 2023-10-06T11:23:18.089 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-06T11:23:18.089 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-06T11:23:18.089 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-06T11:23:18.090 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-06T11:23:18.090 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:23:18.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:18.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43280]: fuse finished with error 0 and tester_r 0 2023-10-06T11:23:18.147 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:23:24.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:23:24.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.249 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:23:24.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:23:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 Processor -- start 2023-10-06T11:23:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 -- start start 2023-10-06T11:23:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f300cc550 0x7f9f300b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f300cabe0 con 0x7f9f301569d0 2023-10-06T11:23:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f300caec0 con 0x7f9f300cc550 2023-10-06T11:23:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f361f0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300cb1a0 con 0x7f9f30158660 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f359ef700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f300cc550 0x7f9f300b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:35462/0 (socket says 172.21.15.7:35462) 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 -- 172.21.15.7:0/1924092826 learned_addr learned my addr 172.21.15.7:0/1924092826 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f351ee700 1 --1- 172.21.15.7:0/1924092826 >> v1:172.21.15.50:6789/0 conn(0x7f9f301569d0 0x7f9f300c82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:46386/0 (socket says 172.21.15.7:46386) 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 -- 172.21.15.7:0/1924092826 >> v2:172.21.15.50:3300/0 conn(0x7f9f300cc550 msgr2=0x7f9f300b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 --2- 172.21.15.7:0/1924092826 >> v2:172.21.15.50:3300/0 conn(0x7f9f300cc550 0x7f9f300b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 -- 172.21.15.7:0/1924092826 >> v1:172.21.15.50:6789/0 conn(0x7f9f301569d0 legacy=0x7f9f300c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.367+0000 7f9f349ed700 1 -- 172.21.15.7:0/1924092826 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300c8ad0 con 0x7f9f30158660 2023-10-06T11:23:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f349ed700 1 --2- 172.21.15.7:0/1924092826 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d0e49cb360d19a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f23fff700 1 -- 172.21.15.7:0/1924092826 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009ea0 con 0x7f9f30158660 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f23fff700 1 -- 172.21.15.7:0/1924092826 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2400b860 con 0x7f9f30158660 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f23fff700 1 -- 172.21.15.7:0/1924092826 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400bc00 con 0x7f9f30158660 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 -- 172.21.15.7:0/1924092826 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 msgr2=0x7f9f30156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 --2- 172.21.15.7:0/1924092826 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24009250 comp rx=0 tx=0).stop 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 -- 172.21.15.7:0/1924092826 shutdown_connections 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 --2- 172.21.15.7:0/1924092826 >> v2:172.21.15.50:3300/0 conn(0x7f9f300cc550 0x7f9f300b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 --2- 172.21.15.7:0/1924092826 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30156da0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 -- 172.21.15.7:0/1924092826 >> 172.21.15.7:0/1924092826 conn(0x7f9f300688a0 msgr2=0x7f9f300cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 -- 172.21.15.7:0/1924092826 shutdown_connections 2023-10-06T11:23:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.371+0000 7f9f361f0700 1 -- 172.21.15.7:0/1924092826 wait complete. 2023-10-06T11:23:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 Processor -- start 2023-10-06T11:23:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 -- start start 2023-10-06T11:23:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f301569d0 0x7f9f30159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30166240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3015afe0 con 0x7f9f300cc550 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3015a130 con 0x7f9f301569d0 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f361f0700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3015a3d0 con 0x7f9f30158660 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f351ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f351ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49658/0 (socket says 172.21.15.7:49658) 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f351ee700 1 -- 172.21.15.7:0/3030579320 learned_addr learned my addr 172.21.15.7:0/3030579320 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f349ed700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f301569d0 0x7f9f30159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.375+0000 7f9f359ef700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30166240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 -- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 msgr2=0x7f9f30166240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30166240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 -- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f301569d0 msgr2=0x7f9f30159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f301569d0 0x7f9f30159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 -- 172.21.15.7:0/3030579320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240094f0 con 0x7f9f300cc550 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f351ee700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c7c0 tx=0x7f9f2c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005a70 con 0x7f9f300cc550 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c005bd0 con 0x7f9f300cc550 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3015a6b0 con 0x7f9f300cc550 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0070d0 con 0x7f9f300cc550 2023-10-06T11:23:24.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3015ac70 con 0x7f9f300cc550 2023-10-06T11:23:24.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.379+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3006c710 con 0x7f9f300cc550 2023-10-06T11:23:24.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.383+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f9f2c007480 con 0x7f9f300cc550 2023-10-06T11:23:24.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.383+0000 7f9f217fa700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 0x7f9f14062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:24.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.383+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f9f2c07c2e0 con 0x7f9f300cc550 2023-10-06T11:23:24.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.383+0000 7f9f349ed700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 0x7f9f14062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:24.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.383+0000 7f9f349ed700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 0x7f9f14062c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24017000 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.391+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f2c04c2b0 con 0x7f9f300cc550 2023-10-06T11:23:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f300b0f80 con 0x7f9f300cc550 2023-10-06T11:23:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f217fa700 1 -- 172.21.15.7:0/3030579320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9f2c04f8d0 con 0x7f9f300cc550 2023-10-06T11:23:24.590 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 msgr2=0x7f9f14062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 0x7f9f14062c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24017000 comp rx=0 tx=0).stop 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 msgr2=0x7f9f3015c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c7c0 tx=0x7f9f2c007f70 comp rx=0 tx=0).stop 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 shutdown_connections 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f9f14060780 0x7f9f14062c30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f9f30158660 0x7f9f30166240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f301569d0 0x7f9f30159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 --2- 172.21.15.7:0/3030579320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f300cc550 0x7f9f3015c9f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 >> 172.21.15.7:0/3030579320 conn(0x7f9f300688a0 msgr2=0x7f9f3006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 shutdown_connections 2023-10-06T11:23:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:24.587+0000 7f9f361f0700 1 -- 172.21.15.7:0/3030579320 wait complete. 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:23:24.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:24.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.606 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-06T11:23:24.613 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:23:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:23:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42497sec preferred_lft 42497sec 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:c9:9f:83:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ba:37:aa:5a:16:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:23:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::68f6:b5ff:fe13:5c60/64 scope link 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ee:6e:ff:5d:f0:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec6e:ffff:fe5d:f075/64 scope link 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ba:37:aa:5a:16:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::b837:aaff:fe5a:1610/64 scope link 2023-10-06T11:23:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:23:24.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.617 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:23:24.630 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-06T11:23:24.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.632 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-06T11:23:24.640 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:23:24.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.641 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-06T11:23:24.642 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-06T11:23:24.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:23:24.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:23:24.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:23:24.642 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-06T11:23:24.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.723 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:23:24.730 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:23:24.731 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:23:24.731 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:24.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:24.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-06T11:23:24.745 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T11:23:24.745 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:23:24.745 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:24.750 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:23:24.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:23:24.750 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:24.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:24.820 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:24.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:24.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:24.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:24.838 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:24.845 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:23:24.845 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T11:23:24.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:24.862 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:24.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:24.921 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:24.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:24.921 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:24.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-06T11:23:24.923+0000 7fa37a974380 -1 init, newargv = 0x55db776a0330 newargc=15 2023-10-06T11:23:24.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43618]: starting ceph client 2023-10-06T11:23:24.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43618]: starting fuse 2023-10-06T11:23:26.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-06T11:23:26.046 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:23:26.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:23:26.104 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:23:26.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:23:26.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:26.106 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-06T11:23:26.112 INFO:teuthology.orchestra.run.smithi007.stdout:52 2023-10-06T11:23:26.112 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T11:23:26.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:23:26.112 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-06T11:23:26.112 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-06T11:23:26.112 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-06T11:23:26.112 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T11:23:26.113 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:26.114 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-06T11:23:26.115 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:23:26.116 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-06T11:23:26.116 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-06T11:23:26.116 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-06T11:23:26.201 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.43618.asok 2023-10-06T11:23:26.387 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43618.asok 2023-10-06T11:23:26.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:26.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43618.asok status 2023-10-06T11:23:26.479 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:26.479 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-06T11:23:26.479 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "43618", 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5517, 2023-10-06T11:23:26.480 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5517 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4188141949 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-06T11:23:26.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4188141949 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5517 192.168.0.1:0/4188141949", 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/4188141949", 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 17, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 40, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-06T11:23:26.482 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:26.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:26.489 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-06T11:23:26.489 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:26.495 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-06T11:23:26.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:26.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:26.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:26.551 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-11_24_00 in ~42s... 2023-10-06T11:23:26.551 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-06T11:23:26.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T11:23:26.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 Processor -- start 2023-10-06T11:23:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 -- start start 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f048415c6a0 0x7f04840c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f048406d800 0x7f04840c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04840c9d40 con 0x7f048415ca70 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04840c9130 con 0x7f048406d800 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f048a8a7700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04840c9410 con 0x7f048415c6a0 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0488e44700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f048406d800 0x7f04840c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f04837fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f048415ca70 0x7f048406d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2023-10-06T11:23:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f04837fe700 1 -- 172.21.15.7:0/1486652504 learned_addr learned my addr 172.21.15.7:0/1486652504 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0483fff700 1 --2- 172.21.15.7:0/1486652504 >> v2:172.21.15.50:3300/0 conn(0x7f048415c6a0 0x7f04840c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0488e44700 1 -- 172.21.15.7:0/1486652504 >> v2:172.21.15.50:3300/0 conn(0x7f048415c6a0 msgr2=0x7f04840c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0488e44700 1 --2- 172.21.15.7:0/1486652504 >> v2:172.21.15.50:3300/0 conn(0x7f048415c6a0 0x7f04840c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0488e44700 1 -- 172.21.15.7:0/1486652504 >> v1:172.21.15.7:6789/0 conn(0x7f048415ca70 legacy=0x7f048406d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0488e44700 1 -- 172.21.15.7:0/1486652504 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840c96f0 con 0x7f048406d800 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.675+0000 7f0482ffd700 1 -- 172.21.15.7:0/1486652504 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434551812 0 0) 0x7f04840c9d40 con 0x7f048415ca70 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f0488e44700 1 --2- 172.21.15.7:0/1486652504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f04840c8ab0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f048406ec90 tx=0x7f047c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fa8b918fda31f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f0482ffd700 1 -- 172.21.15.7:0/1486652504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00e070 con 0x7f048406d800 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f0482ffd700 1 -- 172.21.15.7:0/1486652504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c00bb60 con 0x7f048406d800 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f0482ffd700 1 -- 172.21.15.7:0/1486652504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c012430 con 0x7f048406d800 2023-10-06T11:23:26.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- 172.21.15.7:0/1486652504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 msgr2=0x7f04840c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 --2- 172.21.15.7:0/1486652504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f04840c8ab0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f048406ec90 tx=0x7f047c009780 comp rx=0 tx=0).stop 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- 172.21.15.7:0/1486652504 shutdown_connections 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 --2- 172.21.15.7:0/1486652504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f04840c8ab0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 --2- 172.21.15.7:0/1486652504 >> v2:172.21.15.50:3300/0 conn(0x7f048415c6a0 0x7f04840c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- 172.21.15.7:0/1486652504 >> 172.21.15.7:0/1486652504 conn(0x7f04840ce720 msgr2=0x7f04840ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- 172.21.15.7:0/1486652504 shutdown_connections 2023-10-06T11:23:26.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- 172.21.15.7:0/1486652504 wait complete. 2023-10-06T11:23:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 Processor -- start 2023-10-06T11:23:26.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.679+0000 7f048a8a7700 1 -- start start 2023-10-06T11:23:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f048415c6a0 0x7f04840b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 0x7f04840b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04840b3a20 con 0x7f048406d800 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04840b0e90 con 0x7f048415ca70 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04840b1860 con 0x7f048415c6a0 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49676/0 (socket says 172.21.15.7:49676) 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 -- 172.21.15.7:0/2169295560 learned_addr learned my addr 172.21.15.7:0/2169295560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f04837fe700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f048415c6a0 0x7f04840b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0488e44700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 0x7f04840b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 -- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f048415c6a0 msgr2=0x7f04840b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f048415c6a0 0x7f04840b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 -- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 msgr2=0x7f04840b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 0x7f04840b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 -- 172.21.15.7:0/2169295560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009a20 con 0x7f048406d800 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0488e44700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 0x7f04840b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f0483fff700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f046c00ec80 tx=0x7f046c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00c9d0 con 0x7f048406d800 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00cb30 con 0x7f048406d800 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c013470 con 0x7f048406d800 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04840b3460 con 0x7f048406d800 2023-10-06T11:23:26.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.683+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04840b0b20 con 0x7f048406d800 2023-10-06T11:23:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048404be00 con 0x7f048406d800 2023-10-06T11:23:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f046c013820 con 0x7f048406d800 2023-10-06T11:23:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f047bfff700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 0x7f0464062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f046c07d2d0 con 0x7f048406d800 2023-10-06T11:23:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f04837fe700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 0x7f0464062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.687+0000 7f04837fe700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 0x7f0464062c00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f048416ca90 tx=0x7f0474009040 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.699+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f046c04d2a0 con 0x7f048406d800 2023-10-06T11:23:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.879+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0484157e20 con 0x7f0464060750 2023-10-06T11:23:26.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.895+0000 7f047bfff700 1 -- 172.21.15.7:0/2169295560 <== mgr.5436 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f0484157e20 con 0x7f0464060750 2023-10-06T11:23:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:26.898 INFO:teuthology.orchestra.run.smithi007.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 msgr2=0x7f0464062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 0x7f0464062c00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f048416ca90 tx=0x7f0474009040 comp rx=0 tx=0).stop 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 msgr2=0x7f0484155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f046c00ec80 tx=0x7f046c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 shutdown_connections 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0464060750 0x7f0464062c00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f048415ca70 0x7f04840b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f048415c6a0 0x7f04840b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 --2- 172.21.15.7:0/2169295560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f048406d800 0x7f0484155030 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 >> 172.21.15.7:0/2169295560 conn(0x7f04840ce720 msgr2=0x7f04840cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 shutdown_connections 2023-10-06T11:23:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:26.899+0000 7f048a8a7700 1 -- 172.21.15.7:0/2169295560 wait complete. 2023-10-06T11:23:26.916 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T11:23:26.917 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-06T11:23:26.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T11:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.031+0000 7fcb473e3700 1 Processor -- start 2023-10-06T11:23:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.031+0000 7fcb473e3700 1 -- start start 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb400c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcb400c8b40 0x7fcb400c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb40156210 con 0x7fcb40155e40 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400699c0 con 0x7fcb400c8b40 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c9500 con 0x7fcb400c80d0 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb463e1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb40155e40 0x7fcb40068f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36498/0 (socket says 172.21.15.7:36498) 2023-10-06T11:23:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb463e1700 1 -- 172.21.15.7:0/1186192653 learned_addr learned my addr 172.21.15.7:0/1186192653 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 --2- 172.21.15.7:0/1186192653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb400c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb46be2700 1 --2- 172.21.15.7:0/1186192653 >> v2:172.21.15.50:3300/0 conn(0x7fcb400c8b40 0x7fcb400c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb453df700 1 -- 172.21.15.7:0/1186192653 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116330647 0 0) 0x7fcb40156210 con 0x7fcb40155e40 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb453df700 1 -- 172.21.15.7:0/1186192653 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb2c003570 con 0x7fcb40155e40 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 -- 172.21.15.7:0/1186192653 >> v2:172.21.15.50:3300/0 conn(0x7fcb400c8b40 msgr2=0x7fcb400c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 --2- 172.21.15.7:0/1186192653 >> v2:172.21.15.50:3300/0 conn(0x7fcb400c8b40 0x7fcb400c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 -- 172.21.15.7:0/1186192653 >> v1:172.21.15.7:6789/0 conn(0x7fcb40155e40 legacy=0x7fcb40068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 -- 172.21.15.7:0/1186192653 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400c7c40 con 0x7fcb400c80d0 2023-10-06T11:23:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb45be0700 1 --2- 172.21.15.7:0/1186192653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb400c8540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb300098e0 tx=0x7fcb300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3728c7edd0d06a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb453df700 1 -- 172.21.15.7:0/1186192653 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30011630 con 0x7fcb400c80d0 2023-10-06T11:23:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb453df700 1 -- 172.21.15.7:0/1186192653 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30011790 con 0x7fcb400c80d0 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb453df700 1 -- 172.21.15.7:0/1186192653 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30011b30 con 0x7fcb400c80d0 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- 172.21.15.7:0/1186192653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 msgr2=0x7fcb400c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 --2- 172.21.15.7:0/1186192653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb400c8540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb300098e0 tx=0x7fcb300091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- 172.21.15.7:0/1186192653 shutdown_connections 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 --2- 172.21.15.7:0/1186192653 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb400c8540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 --2- 172.21.15.7:0/1186192653 >> v2:172.21.15.50:3300/0 conn(0x7fcb400c8b40 0x7fcb400c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- 172.21.15.7:0/1186192653 >> 172.21.15.7:0/1186192653 conn(0x7fcb4006ea80 msgr2=0x7fcb4006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- 172.21.15.7:0/1186192653 shutdown_connections 2023-10-06T11:23:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.035+0000 7fcb473e3700 1 -- 172.21.15.7:0/1186192653 wait complete. 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 Processor -- start 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- start start 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb40163e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40155e40 0x7fcb40155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40165020 con 0x7fcb400c8b40 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c6530 con 0x7fcb40155e40 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40155680 con 0x7fcb400c80d0 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49680/0 (socket says 172.21.15.7:49680) 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 -- 172.21.15.7:0/2098992695 learned_addr learned my addr 172.21.15.7:0/2098992695 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb463e1700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb40163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb46be2700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40155e40 0x7fcb40155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 -- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 msgr2=0x7fcb40163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb40163e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 -- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40155e40 msgr2=0x7fcb40155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40155e40 0x7fcb40155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 -- 172.21.15.7:0/2098992695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30009420 con 0x7fcb400c8b40 2023-10-06T11:23:27.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb45be0700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009ed0 tx=0x7fcb30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3001f070 con 0x7fcb400c8b40 2023-10-06T11:23:27.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30011900 con 0x7fcb400c8b40 2023-10-06T11:23:27.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb40164410 con 0x7fcb400c8b40 2023-10-06T11:23:27.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30023410 con 0x7fcb400c8b40 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.039+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb401649d0 con 0x7fcb400c8b40 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb40165190 con 0x7fcb400c8b40 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fcb3002c3b0 con 0x7fcb400c8b40 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb367fc700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 0x7fcb24062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fcb3007e1a0 con 0x7fcb400c8b40 2023-10-06T11:23:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb463e1700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 0x7fcb24062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.043+0000 7fcb463e1700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 0x7fcb24062c30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0058d0 tx=0x7fcb3c009040 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.047+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcb3004e170 con 0x7fcb400c8b40 2023-10-06T11:23:27.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.231+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb4006d720 con 0x7fcb24060780 2023-10-06T11:23:27.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.367+0000 7fcb367fc700 1 -- 172.21.15.7:0/2098992695 <== mgr.5436 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcb4006d720 con 0x7fcb24060780 2023-10-06T11:23:27.371 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule removed for path snap_test_dir1 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 msgr2=0x7fcb24062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 0x7fcb24062c30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0058d0 tx=0x7fcb3c009040 comp rx=0 tx=0).stop 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 msgr2=0x7fcb400c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009ed0 tx=0x7fcb30017000 comp rx=0 tx=0).stop 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 shutdown_connections 2023-10-06T11:23:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fcb24060780 0x7fcb24062c30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40155e40 0x7fcb40155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb400c8b40 0x7fcb400c5ff0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 --2- 172.21.15.7:0/2098992695 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fcb400c80d0 0x7fcb40163e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 >> 172.21.15.7:0/2098992695 conn(0x7fcb4006ea80 msgr2=0x7fcb400abaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 shutdown_connections 2023-10-06T11:23:27.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.371+0000 7fcb473e3700 1 -- 172.21.15.7:0/2098992695 wait complete. 2023-10-06T11:23:27.393 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T11:23:27.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:27.393 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:23:27.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:23:27.412 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T11:23:27.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:23:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.587+0000 7f7b57901700 1 Processor -- start 2023-10-06T11:23:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.587+0000 7f7b57901700 1 -- start start 2023-10-06T11:23:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 0x7f7b500c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5006e900 con 0x7f7b50156be0 2023-10-06T11:23:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5006f4a0 con 0x7f7b50156810 2023-10-06T11:23:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5015a4b0 con 0x7f7b5006a4f0 2023-10-06T11:23:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b54e9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b50156be0 0x7f7b50069de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36506/0 (socket says 172.21.15.7:36506) 2023-10-06T11:23:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b54e9c700 1 -- 172.21.15.7:0/3990614354 learned_addr learned my addr 172.21.15.7:0/3990614354 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b55e9e700 1 --2- 172.21.15.7:0/3990614354 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 0x7f7b500c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670000279 0 0) 0x7f7b5006f4a0 con 0x7f7b50156810 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b3c003570 con 0x7f7b50156810 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658489345 0 0) 0x7f7b5006e900 con 0x7f7b50156be0 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b5006f4a0 con 0x7f7b50156be0 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b55e9e700 1 -- 172.21.15.7:0/3990614354 >> v1:172.21.15.50:6789/0 conn(0x7f7b50156810 legacy=0x7f7b5006ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b55e9e700 1 -- 172.21.15.7:0/3990614354 >> v1:172.21.15.7:6789/0 conn(0x7f7b50156be0 legacy=0x7f7b50069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b55e9e700 1 -- 172.21.15.7:0/3990614354 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5015a1d0 con 0x7f7b5006a4f0 2023-10-06T11:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b55e9e700 1 --2- 172.21.15.7:0/3990614354 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 0x7f7b500c6d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009e90 tx=0x7f7b48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb5f70bde05e4ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800e070 con 0x7f7b5006a4f0 2023-10-06T11:23:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4800bb80 con 0x7f7b5006a4f0 2023-10-06T11:23:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b47fff700 1 -- 172.21.15.7:0/3990614354 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48012430 con 0x7f7b5006a4f0 2023-10-06T11:23:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- 172.21.15.7:0/3990614354 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 msgr2=0x7f7b500c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 --2- 172.21.15.7:0/3990614354 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 0x7f7b500c6d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009e90 tx=0x7f7b48009780 comp rx=0 tx=0).stop 2023-10-06T11:23:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- 172.21.15.7:0/3990614354 shutdown_connections 2023-10-06T11:23:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 --2- 172.21.15.7:0/3990614354 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b5006a4f0 0x7f7b500c6d80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- 172.21.15.7:0/3990614354 >> 172.21.15.7:0/3990614354 conn(0x7f7b500cc2f0 msgr2=0x7f7b500cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- 172.21.15.7:0/3990614354 shutdown_connections 2023-10-06T11:23:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.591+0000 7f7b57901700 1 -- 172.21.15.7:0/3990614354 wait complete. 2023-10-06T11:23:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 Processor -- start 2023-10-06T11:23:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- start start 2023-10-06T11:23:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b50156810 0x7f7b50169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b50156be0 0x7f7b50154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50152760 con 0x7f7b5006a4f0 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50169b80 con 0x7f7b50156810 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5016c8a0 con 0x7f7b50156be0 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49696/0 (socket says 172.21.15.7:49696) 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 -- 172.21.15.7:0/2356839031 learned_addr learned my addr 172.21.15.7:0/2356839031 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b54e9c700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b50156810 0x7f7b50169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b55e9e700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b50156be0 0x7f7b50154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 -- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b50156be0 msgr2=0x7f7b50154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b50156be0 0x7f7b50154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 -- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b50156810 msgr2=0x7f7b50169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b50156810 0x7f7b50169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 -- 172.21.15.7:0/2356839031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48009a20 con 0x7f7b5006a4f0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b5569d700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c790 tx=0x7f7b4c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c010510 con 0x7f7b5006a4f0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c010670 con 0x7f7b5006a4f0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5016cb80 con 0x7f7b5006a4f0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0109f0 con 0x7f7b5006a4f0 2023-10-06T11:23:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.595+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b500af230 con 0x7f7b5006a4f0 2023-10-06T11:23:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.599+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b500af090 con 0x7f7b5006a4f0 2023-10-06T11:23:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.599+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f7b4c012440 con 0x7f7b5006a4f0 2023-10-06T11:23:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.599+0000 7f7b457fa700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 0x7f7b34062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.599+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f7b4c07c5c0 con 0x7f7b5006a4f0 2023-10-06T11:23:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.599+0000 7f7b54e9c700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 0x7f7b34062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.603+0000 7f7b54e9c700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 0x7f7b34062c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b500c3770 tx=0x7f7b40009040 comp rx=0 tx=0).ready entity=mgr.5436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.607+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b4c049be0 con 0x7f7b5006a4f0 2023-10-06T11:23:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:27.807+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f7b5004be00 con 0x7f7b5006a4f0 2023-10-06T11:23:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.499+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f7b4c04ca10 con 0x7f7b5006a4f0 2023-10-06T11:23:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.499+0000 7f7b457fa700 1 -- 172.21.15.7:0/2356839031 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v10) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7b4c0500d0 con 0x7f7b5006a4f0 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 msgr2=0x7f7b34062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 0x7f7b34062c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b500c3770 tx=0x7f7b40009040 comp rx=0 tx=0).stop 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 msgr2=0x7f7b5016c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c790 tx=0x7f7b4c007e30 comp rx=0 tx=0).stop 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 shutdown_connections 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7b34060780 0x7f7b34062c30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7b50156be0 0x7f7b50154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b50156810 0x7f7b50169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 --2- 172.21.15.7:0/2356839031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b5006a4f0 0x7f7b5016c1c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 >> 172.21.15.7:0/2356839031 conn(0x7f7b500cc2f0 msgr2=0x7f7b5006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 shutdown_connections 2023-10-06T11:23:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.503+0000 7f7b57901700 1 -- 172.21.15.7:0/2356839031 wait complete. 2023-10-06T11:23:28.520 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:23:28.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:23:28.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:23:28.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:23:28.556 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-06T11:23:28.556 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:23:28.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T11:23:28.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[43618]: fuse finished with error 0 and tester_r 0 2023-10-06T11:23:28.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:23:28.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:28.595 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:23:28.625 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:28.623+0000 7f3d1dacf000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:23:28.657 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:28.650+0000 7f6932842000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:23:28.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:23:28.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:28.717 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T11:23:28.723 INFO:teuthology.orchestra.run.smithi050.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T11:23:28.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:23:28.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:23:28.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:28.725 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T11:23:28.766 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T11:23:28.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:23:28.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:23:28.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:28.768 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T11:23:28.774 INFO:teuthology.orchestra.run.smithi050.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T11:23:28.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:23:28.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:28.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 Processor -- start 2023-10-06T11:23:28.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 -- start start 2023-10-06T11:23:28.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffb14069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1406aed0 con 0x7ffb140cabd0 2023-10-06T11:23:28.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb14069b10 con 0x7ffb14158c40 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb1933a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14069df0 con 0x7ffb140c8090 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb137fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb140cabd0 0x7ffb140ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36508/0 (socket says 172.21.15.7:36508) 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb137fe700 1 -- 172.21.15.7:0/1613400631 learned_addr learned my addr 172.21.15.7:0/1613400631 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.879+0000 7ffb18b39700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffb14069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136161876 0 0) 0x7ffb1406aed0 con 0x7ffb140cabd0 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003570 con 0x7ffb140cabd0 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547916725 0 0) 0x7ffb14069b10 con 0x7ffb14158c40 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1406aed0 con 0x7ffb14158c40 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1343254808 0 0) 0x7ffafc003570 con 0x7ffb140cabd0 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 msgr2=0x7ffb14069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffb14069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 >> v1:172.21.15.50:6789/0 conn(0x7ffb14158c40 legacy=0x7ffb140ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1406a0d0 con 0x7ffb140cabd0 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7ffb08001470 con 0x7ffb140cabd0 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 >> v1:172.21.15.7:6789/0 conn(0x7ffb140cabd0 legacy=0x7ffb140ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffafc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffafc004180 0x7ffafc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffafc004d50 0x7ffafc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14069df0 con 0x7ffafc004180 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00bba0 con 0x7ffafc004d50 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00bd60 con 0x7ffb140c8090 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffafc004180 0x7ffafc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb13fff700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffafc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb18b39700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffafc004d50 0x7ffafc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 -- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 msgr2=0x7ffafc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c8090 0x7ffafc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 -- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffafc004d50 msgr2=0x7ffafc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffafc004d50 0x7ffafc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 -- 172.21.15.7:0/1613400631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc003570 con 0x7ffafc004180 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb18b39700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffafc004d50 0x7ffafc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb137fe700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffafc004180 0x7ffafc004810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c2d0 tx=0x7ffb08001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7cb8322c52ee4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800cba0 con 0x7ffafc004180 2023-10-06T11:23:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0800cd20 con 0x7ffafc004180 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb12ffd700 1 -- 172.21.15.7:0/1613400631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08005b70 con 0x7ffafc004180 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 -- 172.21.15.7:0/1613400631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffafc004180 msgr2=0x7ffafc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 --2- 172.21.15.7:0/1613400631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffafc004180 0x7ffafc004810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c2d0 tx=0x7ffb08001e10 comp rx=0 tx=0).stop 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb18b39700 1 -- 172.21.15.7:0/1613400631 reap_dead start 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 -- 172.21.15.7:0/1613400631 shutdown_connections 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 -- 172.21.15.7:0/1613400631 >> 172.21.15.7:0/1613400631 conn(0x7ffb1406bd70 msgr2=0x7ffb1406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 -- 172.21.15.7:0/1613400631 shutdown_connections 2023-10-06T11:23:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 -- 172.21.15.7:0/1613400631 wait complete. 2023-10-06T11:23:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.883+0000 7ffb1933a700 1 Processor -- start 2023-10-06T11:23:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- start start 2023-10-06T11:23:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 0x7ffb140ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c67f0 0x7ffb140c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c6cf0 con 0x7ffb14155550 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c6fd0 con 0x7ffb14155180 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c72b0 con 0x7ffb140c67f0 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49724/0 (socket says 172.21.15.7:49724) 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb137fe700 1 -- 172.21.15.7:0/2325280233 learned_addr learned my addr 172.21.15.7:0/2325280233 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 0x7ffb140ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb18b39700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c67f0 0x7ffb140c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c67f0 msgr2=0x7ffb140c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c67f0 0x7ffb140c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 msgr2=0x7ffb140aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 -- 172.21.15.7:0/2325280233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140c7590 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb137fe700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb13fff700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 0x7ffb140ae590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb040010e0 tx=0x7ffb04009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04006a30 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04006b90 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140c7870 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04007050 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140c6090 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.887+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb141691d0 con 0x7ffb14155180 2023-10-06T11:23:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.891+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7ffb04007440 con 0x7ffb14155180 2023-10-06T11:23:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.891+0000 7ffafbfff700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 0x7ffb00062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.891+0000 7ffb137fe700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 msgr2=0x7ffb00062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.891+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7ffb0407ccb0 con 0x7ffb14155180 2023-10-06T11:23:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.891+0000 7ffb137fe700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 0x7ffb00062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:28.899+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffb0404cb50 con 0x7ffb14155180 2023-10-06T11:23:29.074 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.071+0000 7f3d1dacf000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:23:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.095+0000 7ffb137fe700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 msgr2=0x7ffb00062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.095+0000 7ffb137fe700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 0x7ffb00062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:23:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.095+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb1406a3b0 con 0x7ffb14155180 2023-10-06T11:23:29.100 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.094+0000 7f6932842000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:23:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffafbfff700 1 -- 172.21.15.7:0/2325280233 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7ffb0401a030 con 0x7ffb14155180 2023-10-06T11:23:29.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:29.102 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 msgr2=0x7ffb00062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 0x7ffb00062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 msgr2=0x7ffb140ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 0x7ffb140ae590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb040010e0 tx=0x7ffb04009040 comp rx=0 tx=0).stop 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 shutdown_connections 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ffb00060700 0x7ffb00062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ffb140c67f0 0x7ffb140c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb14155550 0x7ffb140aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 --2- 172.21.15.7:0/2325280233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb14155180 0x7ffb140ae590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 >> 172.21.15.7:0/2325280233 conn(0x7ffb1406bd70 msgr2=0x7ffb140b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 shutdown_connections 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.099+0000 7ffb1933a700 1 -- 172.21.15.7:0/2325280233 wait complete. 2023-10-06T11:23:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:29.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:29.125 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.123+0000 7f3d1dacf000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:23:29.153 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.146+0000 7f6932842000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:23:29.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 Processor -- start 2023-10-06T11:23:29.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 -- start start 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c155840 0x7f527c159670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f527c159bb0 0x7f527c0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c0c8c20 con 0x7f527c06c0b0 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c15a610 con 0x7f527c159bb0 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527b59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0c8610 con 0x7f527c155840 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f5279d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f527c06c0b0 0x7f527c0cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36514/0 (socket says 172.21.15.7:36514) 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f5279d9b700 1 -- 172.21.15.7:0/1733261381 learned_addr learned my addr 172.21.15.7:0/1733261381 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527ad9d700 1 --2- 172.21.15.7:0/1733261381 >> v2:172.21.15.7:3300/0 conn(0x7f527c159bb0 0x7f527c0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527a59c700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c155840 0x7f527c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486067284 0 0) 0x7f527c0c8c20 con 0x7f527c06c0b0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5264003570 con 0x7f527c06c0b0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527ad9d700 1 -- 172.21.15.7:0/1733261381 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c155840 msgr2=0x7f527c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527ad9d700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c155840 0x7f527c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527ad9d700 1 -- 172.21.15.7:0/1733261381 >> v1:172.21.15.7:6789/0 conn(0x7f527c06c0b0 legacy=0x7f527c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012342137 0 0) 0x7f5264003570 con 0x7f527c06c0b0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.215+0000 7f527ad9d700 1 -- 172.21.15.7:0/1733261381 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c15a050 con 0x7f527c159bb0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527ad9d700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c90b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5274009e60 tx=0x7f5274009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848e62e9adc86782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400e070 con 0x7f527c159bb0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400b7a0 con 0x7f527c159bb0 2023-10-06T11:23:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527959a700 1 -- 172.21.15.7:0/1733261381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270001470 con 0x7f527c159bb0 2023-10-06T11:23:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- 172.21.15.7:0/1733261381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 msgr2=0x7f527c0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c90b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5274009e60 tx=0x7f5274009780 comp rx=0 tx=0).stop 2023-10-06T11:23:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- 172.21.15.7:0/1733261381 shutdown_connections 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c90b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 --2- 172.21.15.7:0/1733261381 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c155840 0x7f527c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- 172.21.15.7:0/1733261381 >> 172.21.15.7:0/1733261381 conn(0x7f527c06f380 msgr2=0x7f527c06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- 172.21.15.7:0/1733261381 shutdown_connections 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- 172.21.15.7:0/1733261381 wait complete. 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 Processor -- start 2023-10-06T11:23:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.219+0000 7f527b59e700 1 -- start start 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527c155840 0x7f527c145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c0c55e0 0x7f527c157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0c4920 con 0x7f527c159bb0 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0c3ac0 con 0x7f527c155840 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0c9350 con 0x7f527c0c55e0 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49750/0 (socket says 172.21.15.7:49750) 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 -- 172.21.15.7:0/1562299940 learned_addr learned my addr 172.21.15.7:0/1562299940 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527a59c700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527c155840 0x7f527c145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527ad9d700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c0c55e0 0x7f527c157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 -- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c0c55e0 msgr2=0x7f527c157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c0c55e0 0x7f527c157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 -- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527c155840 msgr2=0x7f527c145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527c155840 0x7f527c145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 -- 172.21.15.7:0/1562299940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274009a20 con 0x7f527c159bb0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f5279d9b700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f527000eb70 tx=0x7f5270001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000cb20 con 0x7f527c159bb0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527000cca0 con 0x7f527c159bb0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c0c3d10 con 0x7f527c159bb0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700073a0 con 0x7f527c159bb0 2023-10-06T11:23:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c0c42d0 con 0x7f527c159bb0 2023-10-06T11:23:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c0af2d0 con 0x7f527c159bb0 2023-10-06T11:23:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.223+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f5270004830 con 0x7f527c159bb0 2023-10-06T11:23:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.227+0000 7f526a7fc700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 0x7f525c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.227+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f527007f810 con 0x7f527c159bb0 2023-10-06T11:23:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.227+0000 7f527a59c700 1 -- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 msgr2=0x7f525c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.227+0000 7f527a59c700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 0x7f525c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.231+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f527c0af2d0 con 0x7f527c159bb0 2023-10-06T11:23:29.241 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.239+0000 7f3d1dacf000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:29.263 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.258+0000 7f6932842000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.391+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527c0af2d0 con 0x7f527c159bb0 2023-10-06T11:23:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.391+0000 7f526a7fc700 1 -- 172.21.15.7:0/1562299940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17341 (secure 0 0 0) 0x7f527c0af2d0 con 0x7f527c159bb0 2023-10-06T11:23:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:29.397 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:16.399561+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/2634621881":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3120583769":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/330210624":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3694782510":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6833/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6832/33317":"2023-10-07T11:23:16.399486+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 msgr2=0x7f525c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 0x7f525c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 msgr2=0x7f527c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f527000eb70 tx=0x7f5270001b80 comp rx=0 tx=0).stop 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 shutdown_connections 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f525c060700 0x7f525c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f527c0c55e0 0x7f527c157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f527c159bb0 0x7f527c0c50a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 --2- 172.21.15.7:0/1562299940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527c155840 0x7f527c145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 >> 172.21.15.7:0/1562299940 conn(0x7f527c06f380 msgr2=0x7f527c0c17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 shutdown_connections 2023-10-06T11:23:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.395+0000 7f527b59e700 1 -- 172.21.15.7:0/1562299940 wait complete. 2023-10-06T11:23:29.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:21:50.897583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:21:51.875482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:23:00.705486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:23:01.914689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:23:29.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:23:29.447 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.443+0000 7f3d1dacf000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:23:29.465 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.458+0000 7f6932842000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:23:29.500 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.495+0000 7f3d1dacf000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:29.515 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.510+0000 7f6932842000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 Processor -- start 2023-10-06T11:23:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 -- start start 2023-10-06T11:23:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155860 0x7fde080c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde0806bfb0 0x7fde080c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde080c9a60 con 0x7fde08155c30 2023-10-06T11:23:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde080caea0 con 0x7fde0806bfb0 2023-10-06T11:23:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0dc91700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080c9c90 con 0x7fde08155860 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde0806bfb0 0x7fde080c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde0806bfb0 0x7fde080c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49764/0 (socket says 172.21.15.7:49764) 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 -- 172.21.15.7:0/292330643 learned_addr learned my addr 172.21.15.7:0/292330643 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0cc8f700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155860 0x7fde080c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 -- 172.21.15.7:0/292330643 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155860 msgr2=0x7fde080c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155860 0x7fde080c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 -- 172.21.15.7:0/292330643 >> v1:172.21.15.50:6789/0 conn(0x7fde08155c30 legacy=0x7fde0806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 -- 172.21.15.7:0/292330643 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080ca570 con 0x7fde0806bfb0 2023-10-06T11:23:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde0d490700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080c9520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde00009f20 tx=0x7fde00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd49c6d1f83c00e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde077fe700 1 -- 172.21.15.7:0/292330643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000e070 con 0x7fde0806bfb0 2023-10-06T11:23:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde077fe700 1 -- 172.21.15.7:0/292330643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000ba80 con 0x7fde0806bfb0 2023-10-06T11:23:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.547+0000 7fde077fe700 1 -- 172.21.15.7:0/292330643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000092e0 con 0x7fde0806bfb0 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- 172.21.15.7:0/292330643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 msgr2=0x7fde080c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080c9520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde00009f20 tx=0x7fde00009840 comp rx=0 tx=0).stop 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- 172.21.15.7:0/292330643 shutdown_connections 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080c9520 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- 172.21.15.7:0/292330643 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155860 0x7fde080c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- 172.21.15.7:0/292330643 >> 172.21.15.7:0/292330643 conn(0x7fde0806f2a0 msgr2=0x7fde08068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- 172.21.15.7:0/292330643 shutdown_connections 2023-10-06T11:23:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- 172.21.15.7:0/292330643 wait complete. 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 Processor -- start 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- start start 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde08155860 0x7fde0814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 0x7fde080c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080b2370 con 0x7fde0806bfb0 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0816cae0 con 0x7fde08155860 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0dc91700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080b18c0 con 0x7fde08155c30 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0cc8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0cc8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49766/0 (socket says 172.21.15.7:49766) 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0cc8f700 1 -- 172.21.15.7:0/290861668 learned_addr learned my addr 172.21.15.7:0/290861668 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde07fff700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde08155860 0x7fde0814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0d490700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 0x7fde080c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0cc8f700 1 -- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 msgr2=0x7fde080c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.551+0000 7fde0cc8f700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 0x7fde080c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0cc8f700 1 -- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde08155860 msgr2=0x7fde0814ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0cc8f700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde08155860 0x7fde0814ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0cc8f700 1 -- 172.21.15.7:0/290861668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00009ae0 con 0x7fde0806bfb0 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0d490700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 0x7fde080c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0cc8f700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fddf800db90 tx=0x7fddf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800a8a0 con 0x7fde0806bfb0 2023-10-06T11:23:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0816c490 con 0x7fde0806bfb0 2023-10-06T11:23:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800aa00 con 0x7fde0806bfb0 2023-10-06T11:23:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800adc0 con 0x7fde0806bfb0 2023-10-06T11:23:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde080b1ba0 con 0x7fde0806bfb0 2023-10-06T11:23:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0806da60 con 0x7fde0806bfb0 2023-10-06T11:23:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.555+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fddf8019690 con 0x7fde0806bfb0 2023-10-06T11:23:29.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.559+0000 7fde04ff9700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 0x7fddf4062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.559+0000 7fde07fff700 1 -- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 msgr2=0x7fddf4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:29.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.559+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fddf8014070 con 0x7fde0806bfb0 2023-10-06T11:23:29.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.559+0000 7fde07fff700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 0x7fddf4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.567+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fddf804d700 con 0x7fde0806bfb0 2023-10-06T11:23:29.661 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.659+0000 7f3d1dacf000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:23:29.670 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.666+0000 7f6932842000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:23:29.721 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.715+0000 7f3d1dacf000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:23:29.728 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.722+0000 7f6932842000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:23:29.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.723+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde0816c910 con 0x7fde0806bfb0 2023-10-06T11:23:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.723+0000 7fde04ff9700 1 -- 172.21.15.7:0/290861668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fddf8050d20 con 0x7fde0806bfb0 2023-10-06T11:23:29.729 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-10-06T11:23:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 msgr2=0x7fddf4062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 0x7fddf4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 msgr2=0x7fde080b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fddf800db90 tx=0x7fddf8008040 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 shutdown_connections 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fddf4060700 0x7fddf4062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde08155c30 0x7fde080c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde08155860 0x7fde0814ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 --2- 172.21.15.7:0/290861668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0806bfb0 0x7fde080b1e30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 >> 172.21.15.7:0/290861668 conn(0x7fde0806f2a0 msgr2=0x7fde08151190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 shutdown_connections 2023-10-06T11:23:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.727+0000 7fde0dc91700 1 -- 172.21.15.7:0/290861668 wait complete. 2023-10-06T11:23:29.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:23:29.779 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:29.775+0000 7f3d1dacf000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:23:29.788 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:29.782+0000 7f6932842000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:23:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 Processor -- start 2023-10-06T11:23:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- start start 2023-10-06T11:23:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff498158c60 0x7ff49806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498159030 0x7ff49806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980c7980 con 0x7ff498067e30 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49806cd70 con 0x7ff498158c60 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49806d050 con 0x7ff498159030 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff498158c60 0x7ff49806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff497fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff498067e30 0x7ff49815c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36518/0 (socket says 172.21.15.7:36518) 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff497fff700 1 -- 172.21.15.7:0/2834231439 learned_addr learned my addr 172.21.15.7:0/2834231439 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff48effd700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498159030 0x7ff49806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 -- 172.21.15.7:0/2834231439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498159030 msgr2=0x7ff49806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498159030 0x7ff49806c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff496ffd700 1 -- 172.21.15.7:0/2834231439 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527593534 0 0) 0x7ff4980c7980 con 0x7ff498067e30 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 -- 172.21.15.7:0/2834231439 >> v1:172.21.15.7:6789/0 conn(0x7ff498067e30 legacy=0x7ff49815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 -- 172.21.15.7:0/2834231439 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980c7610 con 0x7ff498158c60 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff4977fe700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498158c60 0x7ff49806c250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4162a69961c5bce4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff496ffd700 1 -- 172.21.15.7:0/2834231439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488009e80 con 0x7ff498158c60 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff496ffd700 1 -- 172.21.15.7:0/2834231439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48800b860 con 0x7ff498158c60 2023-10-06T11:23:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff496ffd700 1 -- 172.21.15.7:0/2834231439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800bbe0 con 0x7ff498158c60 2023-10-06T11:23:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- 172.21.15.7:0/2834231439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498158c60 msgr2=0x7ff49806c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498158c60 0x7ff49806c250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff4880091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- 172.21.15.7:0/2834231439 shutdown_connections 2023-10-06T11:23:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498159030 0x7ff49806c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/2834231439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498158c60 0x7ff49806c250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- 172.21.15.7:0/2834231439 >> 172.21.15.7:0/2834231439 conn(0x7ff49806f2a0 msgr2=0x7ff498068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- 172.21.15.7:0/2834231439 shutdown_connections 2023-10-06T11:23:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.851+0000 7ff49c9a6700 1 -- 172.21.15.7:0/2834231439 wait complete. 2023-10-06T11:23:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 Processor -- start 2023-10-06T11:23:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 -- start start 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498158c60 0x7ff4980a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980ab620 0x7ff4980ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980c3a50 con 0x7ff498067e30 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980a9100 con 0x7ff4980ab620 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff49c9a6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980c3ca0 con 0x7ff498158c60 2023-10-06T11:23:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49780/0 (socket says 172.21.15.7:49780) 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 -- 172.21.15.7:0/458222219 learned_addr learned my addr 172.21.15.7:0/458222219 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff48effd700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498158c60 0x7ff4980a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff497fff700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980ab620 0x7ff4980ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 -- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498158c60 msgr2=0x7ff4980a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498158c60 0x7ff4980a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 -- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980ab620 msgr2=0x7ff4980ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980ab620 0x7ff4980ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 -- 172.21.15.7:0/458222219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009420 con 0x7ff498067e30 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff4977fe700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff498071180 tx=0x7ff488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.855+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800bbe0 con 0x7ff498067e30 2023-10-06T11:23:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488000d10 con 0x7ff498067e30 2023-10-06T11:23:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4980c3f80 con 0x7ff498067e30 2023-10-06T11:23:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488011690 con 0x7ff498067e30 2023-10-06T11:23:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4980a8df0 con 0x7ff498067e30 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49806da60 con 0x7ff498067e30 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7ff488011a00 con 0x7ff498067e30 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48ffff700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 0x7ff478062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48effd700 1 -- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 msgr2=0x7ff478062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7ff48807e780 con 0x7ff498067e30 2023-10-06T11:23:29.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.859+0000 7ff48effd700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 0x7ff478062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:29.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:29.867+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff48804e750 con 0x7ff498067e30 2023-10-06T11:23:30.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.031+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4980c7300 con 0x7ff498067e30 2023-10-06T11:23:30.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff48ffff700 1 -- 172.21.15.7:0/458222219 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff488011cb0 con 0x7ff498067e30 2023-10-06T11:23:30.037 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-06T11:23:30.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 msgr2=0x7ff478062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 0x7ff478062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 msgr2=0x7ff4980c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff498071180 tx=0x7ff488017000 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 shutdown_connections 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff478060730 0x7ff478062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4980ab620 0x7ff4980ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff498158c60 0x7ff4980a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 --2- 172.21.15.7:0/458222219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff498067e30 0x7ff4980c6450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 >> 172.21.15.7:0/458222219 conn(0x7ff49806f2a0 msgr2=0x7ff4980c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 shutdown_connections 2023-10-06T11:23:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.035+0000 7ff49c9a6700 1 -- 172.21.15.7:0/458222219 wait complete. 2023-10-06T11:23:30.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:23:30.094 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.090+0000 7f6932842000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:23:30.121 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.119+0000 7f3d1dacf000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:23:30.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 Processor -- start 2023-10-06T11:23:30.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 -- start start 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7278155940 0x7f72781597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7278156620 con 0x7f7278155d10 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7278159ff0 con 0x7f727806e110 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f727d391700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727815a2d0 con 0x7f7278155940 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.155+0000 7f72777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7278155d10 0x7f72780c7f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36530/0 (socket says 172.21.15.7:36530) 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f72777fe700 1 -- 172.21.15.7:0/402264345 learned_addr learned my addr 172.21.15.7:0/402264345 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:30.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7276ffd700 1 --2- 172.21.15.7:0/402264345 >> v2:172.21.15.50:3300/0 conn(0x7f7278155940 0x7f72781597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675758585 0 0) 0x7f7278156620 con 0x7f7278155d10 2023-10-06T11:23:30.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7260003570 con 0x7f7278155d10 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380356963 0 0) 0x7f7278159ff0 con 0x7f727806e110 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7278156620 con 0x7f727806e110 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7276ffd700 1 -- 172.21.15.7:0/402264345 >> v1:172.21.15.50:6789/0 conn(0x7f727806e110 legacy=0x7f7278156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7276ffd700 1 -- 172.21.15.7:0/402264345 >> v1:172.21.15.7:6789/0 conn(0x7f7278155d10 legacy=0x7f72780c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7276ffd700 1 -- 172.21.15.7:0/402264345 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815a5b0 con 0x7f7278155940 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7276ffd700 1 --2- 172.21.15.7:0/402264345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7278155940 0x7f72781597a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72680098e0 tx=0x7f72680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896e206b146e7871 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680108a0 con 0x7f7278155940 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268010a00 con 0x7f7278155940 2023-10-06T11:23:30.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f7275ffb700 1 -- 172.21.15.7:0/402264345 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268010da0 con 0x7f7278155940 2023-10-06T11:23:30.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 -- 172.21.15.7:0/402264345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7278155940 msgr2=0x7f72781597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 --2- 172.21.15.7:0/402264345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7278155940 0x7f72781597a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72680098e0 tx=0x7f72680091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 -- 172.21.15.7:0/402264345 shutdown_connections 2023-10-06T11:23:30.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 --2- 172.21.15.7:0/402264345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7278155940 0x7f72781597a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 -- 172.21.15.7:0/402264345 >> 172.21.15.7:0/402264345 conn(0x7f72780cc2f0 msgr2=0x7f72780cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 -- 172.21.15.7:0/402264345 shutdown_connections 2023-10-06T11:23:30.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.159+0000 7f727d391700 1 -- 172.21.15.7:0/402264345 wait complete. 2023-10-06T11:23:30.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 Processor -- start 2023-10-06T11:23:30.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 -- start start 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f727806e110 0x7f72780c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7278155940 0x7f7278165ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780c5520 con 0x7f7278155d10 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278166000 con 0x7f727806e110 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f727d391700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72780c4910 con 0x7f7278155940 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f7276ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f727806e110 0x7f72780c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49782/0 (socket says 172.21.15.7:49782) 2023-10-06T11:23:30.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 -- 172.21.15.7:0/3221415227 learned_addr learned my addr 172.21.15.7:0/3221415227 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:30.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72767fc700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7278155940 0x7f7278165ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 -- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7278155940 msgr2=0x7f7278165ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7278155940 0x7f7278165ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 -- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f727806e110 msgr2=0x7f72780c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f727806e110 0x7f72780c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 -- 172.21.15.7:0/3221415227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268009420 con 0x7f7278155d10 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f72777fe700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f727000cbc0 tx=0x7f7270001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000a7e0 con 0x7f7278155d10 2023-10-06T11:23:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.163+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727000a940 con 0x7f7278155d10 2023-10-06T11:23:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72780c4bf0 con 0x7f7278155d10 2023-10-06T11:23:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000ad00 con 0x7f7278155d10 2023-10-06T11:23:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72780c51b0 con 0x7f7278155d10 2023-10-06T11:23:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72780c56d0 con 0x7f7278155d10 2023-10-06T11:23:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f7270007300 con 0x7f7278155d10 2023-10-06T11:23:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f725effd700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 0x7f7264062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f727007d0d0 con 0x7f7278155d10 2023-10-06T11:23:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f7276ffd700 1 -- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 msgr2=0x7f7264062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.167+0000 7f7276ffd700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 0x7f7264062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.171+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f72780c56d0 con 0x7f7278155d10 2023-10-06T11:23:30.208 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.202+0000 7f6932842000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:23:30.240 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.235+0000 7f3d1dacf000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:23:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.327+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f727804be00 con 0x7f7278155d10 2023-10-06T11:23:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f725effd700 1 -- 172.21.15.7:0/3221415227 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f727004d120 con 0x7f7278155d10 2023-10-06T11:23:30.334 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-06T11:23:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 msgr2=0x7f7264062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 0x7f7264062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 msgr2=0x7f7278158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f727000cbc0 tx=0x7f7270001a20 comp rx=0 tx=0).stop 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 shutdown_connections 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f7264060700 0x7f7264062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7278155d10 0x7f7278158af0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f7278155940 0x7f7278165ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 --2- 172.21.15.7:0/3221415227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f727806e110 0x7f72780c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 >> 172.21.15.7:0/3221415227 conn(0x7f72780cc2f0 msgr2=0x7f727806b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 shutdown_connections 2023-10-06T11:23:30.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.331+0000 7f727d391700 1 -- 172.21.15.7:0/3221415227 wait complete. 2023-10-06T11:23:30.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:23:30.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 Processor -- start 2023-10-06T11:23:30.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 -- start start 2023-10-06T11:23:30.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6940acdc0 0x7fc6940cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940cc3b0 con 0x7fc6940ac9f0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc694157320 con 0x7fc6940acdc0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc693fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940cc690 con 0x7fc69406d6e0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc6937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6940acdc0 0x7fc6940cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc692ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6940ac9f0 0x7fc69406cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36536/0 (socket says 172.21.15.7:36536) 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.451+0000 7fc692ffd700 1 -- 172.21.15.7:0/2054441568 learned_addr learned my addr 172.21.15.7:0/2054441568 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6927fc700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689151735 0 0) 0x7fc6940cc3b0 con 0x7fc6940ac9f0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c003570 con 0x7fc6940ac9f0 2023-10-06T11:23:30.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6937fe700 1 -- 172.21.15.7:0/2054441568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 msgr2=0x7fc6940cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6937fe700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6937fe700 1 -- 172.21.15.7:0/2054441568 >> v1:172.21.15.7:6789/0 conn(0x7fc6940ac9f0 legacy=0x7fc69406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867824115 0 0) 0x7fc67c003570 con 0x7fc6940ac9f0 2023-10-06T11:23:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6937fe700 1 -- 172.21.15.7:0/2054441568 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940cc970 con 0x7fc6940acdc0 2023-10-06T11:23:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc6937fe700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6940cb6a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e60 tx=0x7fc68c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744396b4b1a9eed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00e070 con 0x7fc6940acdc0 2023-10-06T11:23:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00b7a0 con 0x7fc6940acdc0 2023-10-06T11:23:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc691ffb700 1 -- 172.21.15.7:0/2054441568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684001470 con 0x7fc6940acdc0 2023-10-06T11:23:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 -- 172.21.15.7:0/2054441568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 msgr2=0x7fc6940cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6940cb6a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e60 tx=0x7fc68c009780 comp rx=0 tx=0).stop 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 -- 172.21.15.7:0/2054441568 shutdown_connections 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6940cb6a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 --2- 172.21.15.7:0/2054441568 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 -- 172.21.15.7:0/2054441568 >> 172.21.15.7:0/2054441568 conn(0x7fc6940ce570 msgr2=0x7fc6940ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 -- 172.21.15.7:0/2054441568 shutdown_connections 2023-10-06T11:23:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 -- 172.21.15.7:0/2054441568 wait complete. 2023-10-06T11:23:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.455+0000 7fc693fff700 1 Processor -- start 2023-10-06T11:23:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- start start 2023-10-06T11:23:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6940ac9f0 0x7fc6940b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940b0dc0 con 0x7fc6940acdc0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940b10a0 con 0x7fc6940ac9f0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940b1380 con 0x7fc69406d6e0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49800/0 (socket says 172.21.15.7:49800) 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 -- 172.21.15.7:0/3549131840 learned_addr learned my addr 172.21.15.7:0/3549131840 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc692ffd700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6927fc700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6940ac9f0 0x7fc6940b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 -- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 msgr2=0x7fc6940b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 -- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6940ac9f0 msgr2=0x7fc6940b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6940ac9f0 0x7fc6940b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 -- 172.21.15.7:0/3549131840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009a20 con 0x7fc6940acdc0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc694063870 tx=0x7fc68c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c012c70 con 0x7fc6940acdc0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c012e10 con 0x7fc6940acdc0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c01d680 con 0x7fc6940acdc0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6940b1660 con 0x7fc6940acdc0 2023-10-06T11:23:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc694145a30 con 0x7fc6940acdc0 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.459+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69406eb50 con 0x7fc6940acdc0 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.463+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc68c024070 con 0x7fc6940acdc0 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.463+0000 7fc67affd700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 0x7fc680062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.463+0000 7fc692ffd700 1 -- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 msgr2=0x7fc680062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.463+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fc68c07f070 con 0x7fc6940acdc0 2023-10-06T11:23:30.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.463+0000 7fc692ffd700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 0x7fc680062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:30.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.471+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc68c04f040 con 0x7fc6940acdc0 2023-10-06T11:23:30.499 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.494+0000 7f6932842000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:23:30.558 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.555+0000 7f3d1dacf000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:23:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6940ccc50 con 0x7fc6940acdc0 2023-10-06T11:23:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc67affd700 1 -- 172.21.15.7:0/3549131840 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc68c052660 con 0x7fc6940acdc0 2023-10-06T11:23:30.638 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-06T11:23:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 msgr2=0x7fc680062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 0x7fc680062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 msgr2=0x7fc6941454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.635+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc694063870 tx=0x7fc68c018000 comp rx=0 tx=0).stop 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 shutdown_connections 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fc680060700 0x7fc680062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6940acdc0 0x7fc6941454f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6940ac9f0 0x7fc6940b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 --2- 172.21.15.7:0/3549131840 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fc69406d6e0 0x7fc6940b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 >> 172.21.15.7:0/3549131840 conn(0x7fc6940ce570 msgr2=0x7fc6940cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 shutdown_connections 2023-10-06T11:23:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.639+0000 7fc693fff700 1 -- 172.21.15.7:0/3549131840 wait complete. 2023-10-06T11:23:30.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:30.665 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.658+0000 7f6932842000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:30.729 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.723+0000 7f3d1dacf000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:23:30.732 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.726+0000 7f6932842000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:23:30.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 Processor -- start 2023-10-06T11:23:30.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- start start 2023-10-06T11:23:30.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdb806a3e0 0x7fbdb806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80abd70 con 0x7fbdb80c71e0 2023-10-06T11:23:30.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb806c090 con 0x7fbdb8157970 2023-10-06T11:23:30.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80c6dd0 con 0x7fbdb806a3e0 2023-10-06T11:23:30.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdb806a3e0 0x7fbdb806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb5d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbdb80c71e0 0x7fbdb8069cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36544/0 (socket says 172.21.15.7:36544) 2023-10-06T11:23:30.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb5d9b700 1 -- 172.21.15.7:0/2971263089 learned_addr learned my addr 172.21.15.7:0/2971263089 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057871309 0 0) 0x7fbdb80abd70 con 0x7fbdb80c71e0 2023-10-06T11:23:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda0003570 con 0x7fbdb80c71e0 2023-10-06T11:23:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225920476 0 0) 0x7fbdb806c090 con 0x7fbdb8157970 2023-10-06T11:23:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2971263089 >> v1:172.21.15.50:6789/0 conn(0x7fbdb8157970 legacy=0x7fbdb8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2971263089 >> v1:172.21.15.7:6789/0 conn(0x7fbdb80c71e0 legacy=0x7fbdb8069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2971263089 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80c7670 con 0x7fbdb806a3e0 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb6d9d700 1 --2- 172.21.15.7:0/2971263089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb806a3e0 0x7fbdb806e000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e60 tx=0x7fbdb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304ef0f65c20c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000e070 con 0x7fbdb806a3e0 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000bbf0 con 0x7fbdb806a3e0 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb559a700 1 -- 172.21.15.7:0/2971263089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0012430 con 0x7fbdb806a3e0 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- 172.21.15.7:0/2971263089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb806a3e0 msgr2=0x7fbdb806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2971263089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb806a3e0 0x7fbdb806e000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e60 tx=0x7fbdb0009780 comp rx=0 tx=0).stop 2023-10-06T11:23:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- 172.21.15.7:0/2971263089 shutdown_connections 2023-10-06T11:23:30.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2971263089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb806a3e0 0x7fbdb806e000 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.751+0000 7fbdb759e700 1 -- 172.21.15.7:0/2971263089 >> 172.21.15.7:0/2971263089 conn(0x7fbdb80cc190 msgr2=0x7fbdb80cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- 172.21.15.7:0/2971263089 shutdown_connections 2023-10-06T11:23:30.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- 172.21.15.7:0/2971263089 wait complete. 2023-10-06T11:23:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 Processor -- start 2023-10-06T11:23:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- start start 2023-10-06T11:23:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb806a3e0 0x7fbdb81540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbdb80c71e0 0x7fbdb8150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b0810 con 0x7fbdb8157970 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b0a60 con 0x7fbdb806a3e0 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b0d40 con 0x7fbdb80c71e0 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49818/0 (socket says 172.21.15.7:49818) 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2123011560 learned_addr learned my addr 172.21.15.7:0/2123011560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb659c700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb806a3e0 0x7fbdb81540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb5d9b700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbdb80c71e0 0x7fbdb8150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbdb80c71e0 msgr2=0x7fbdb8150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbdb80c71e0 0x7fbdb8150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb806a3e0 msgr2=0x7fbdb81540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb806a3e0 0x7fbdb81540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 -- 172.21.15.7:0/2123011560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009a20 con 0x7fbdb8157970 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb6d9d700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b6c0 tx=0x7fbdb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0027470 con 0x7fbdb8157970 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0027610 con 0x7fbdb8157970 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb80b1020 con 0x7fbdb8157970 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00279d0 con 0x7fbdb8157970 2023-10-06T11:23:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.755+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb80b1820 con 0x7fbdb8157970 2023-10-06T11:23:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb80aea80 con 0x7fbdb8157970 2023-10-06T11:23:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fbdb0024070 con 0x7fbdb8157970 2023-10-06T11:23:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbda67fc700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 0x7fbd98062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fbdb007f800 con 0x7fbdb8157970 2023-10-06T11:23:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbdb659c700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 msgr2=0x7fbd98062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.759+0000 7fbdb659c700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 0x7fbd98062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.767+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbdb004f730 con 0x7fbdb8157970 2023-10-06T11:23:30.787 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.782+0000 7f6932842000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:23:30.794 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.791+0000 7f3d1dacf000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:23:30.841 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:30.834+0000 7f6932842000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:23:30.846 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.843+0000 7f3d1dacf000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:23:30.902 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:30.899+0000 7f3d1dacf000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:23:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.959+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdb806ba40 con 0x7fbdb8157970 2023-10-06T11:23:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.959+0000 7fbda67fc700 1 -- 172.21.15.7:0/2123011560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7fbdb0052d50 con 0x7fbdb8157970 2023-10-06T11:23:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:30.963 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 msgr2=0x7fbd98062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb659c700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 msgr2=0x7fbd98062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb659c700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 0x7fbd98062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 0x7fbd98062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 msgr2=0x7fbdb80aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b6c0 tx=0x7fbdb0019000 comp rx=0 tx=0).stop 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 shutdown_connections 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fbd98060700 0x7fbd98062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb8157970 0x7fbdb80aef70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fbdb80c71e0 0x7fbdb8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 --2- 172.21.15.7:0/2123011560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb806a3e0 0x7fbdb81540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 >> 172.21.15.7:0/2123011560 conn(0x7fbdb80cc190 msgr2=0x7fbdb8154cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 shutdown_connections 2023-10-06T11:23:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:30.963+0000 7fbdb759e700 1 -- 172.21.15.7:0/2123011560 wait complete. 2023-10-06T11:23:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:30.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:31.017 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.010+0000 7f6932842000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:31.072 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.066+0000 7f6932842000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:23:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 Processor -- start 2023-10-06T11:23:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 -- start start 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 0x7f0a7c069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0a7c0c8090 0x7f0a7c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c06aed0 con 0x7f0a7c0c8090 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c069b10 con 0x7f0a7c158c40 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a82f6c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c069df0 con 0x7f0a7c0ca4c0 2023-10-06T11:23:31.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0a7c0c8090 0x7f0a7c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0a7c0c8090 0x7f0a7c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49824/0 (socket says 172.21.15.7:49824) 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 -- 172.21.15.7:0/193835377 learned_addr learned my addr 172.21.15.7:0/193835377 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a8276b700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81f6a700 1 --2- 172.21.15.7:0/193835377 >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 0x7f0a7c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 -- 172.21.15.7:0/193835377 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 msgr2=0x7f0a7c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 -- 172.21.15.7:0/193835377 >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 msgr2=0x7f0a7c069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 --2- 172.21.15.7:0/193835377 >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 0x7f0a7c069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81769700 1 -- 172.21.15.7:0/193835377 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c06a0d0 con 0x7f0a7c0c8090 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a8276b700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.079+0000 7f0a81f6a700 1 --2- 172.21.15.7:0/193835377 >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 0x7f0a7c069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a81769700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c0c8090 0x7f0a7c159010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c06eb90 tx=0x7f0a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2448b1ebb01164b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a80f68700 1 -- 172.21.15.7:0/193835377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00b860 con 0x7f0a7c0c8090 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a80f68700 1 -- 172.21.15.7:0/193835377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c00b9c0 con 0x7f0a7c0c8090 2023-10-06T11:23:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a80f68700 1 -- 172.21.15.7:0/193835377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00bd60 con 0x7f0a7c0c8090 2023-10-06T11:23:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- 172.21.15.7:0/193835377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c0c8090 msgr2=0x7f0a7c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c0c8090 0x7f0a7c159010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c06eb90 tx=0x7f0a6c009250 comp rx=0 tx=0).stop 2023-10-06T11:23:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- 172.21.15.7:0/193835377 shutdown_connections 2023-10-06T11:23:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/193835377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c0c8090 0x7f0a7c159010 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/193835377 >> v2:172.21.15.50:3300/0 conn(0x7f0a7c158c40 0x7f0a7c069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- 172.21.15.7:0/193835377 >> 172.21.15.7:0/193835377 conn(0x7f0a7c06bd70 msgr2=0x7f0a7c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- 172.21.15.7:0/193835377 shutdown_connections 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- 172.21.15.7:0/193835377 wait complete. 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 Processor -- start 2023-10-06T11:23:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.083+0000 7f0a82f6c700 1 -- start start 2023-10-06T11:23:31.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0c8090 0x7f0a7c155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c15e930 con 0x7f0a7c158c40 2023-10-06T11:23:31.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c15dd20 con 0x7f0a7c0ca4c0 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c15e000 con 0x7f0a7c0c8090 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49838/0 (socket says 172.21.15.7:49838) 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 -- 172.21.15.7:0/4155010451 learned_addr learned my addr 172.21.15.7:0/4155010451 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a81769700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a81f6a700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0c8090 0x7f0a7c155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 -- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0c8090 msgr2=0x7f0a7c155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0c8090 0x7f0a7c155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 -- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7c0ca4c0 msgr2=0x7f0a7c0afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 -- 172.21.15.7:0/4155010451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0094f0 con 0x7f0a7c158c40 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a8276b700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cf60 tx=0x7f0a74007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400ae40 con 0x7f0a7c158c40 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74015440 con 0x7f0a7c158c40 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c15e2e0 con 0x7f0a7c158c40 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74015800 con 0x7f0a7c158c40 2023-10-06T11:23:31.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c156530 con 0x7f0a7c158c40 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.087+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c06ec60 con 0x7f0a7c158c40 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.091+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f0a7400d440 con 0x7f0a7c158c40 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.091+0000 7f0a71ffb700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 0x7f0a60062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.091+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f0a7407da50 con 0x7f0a7c158c40 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.091+0000 7f0a81f6a700 1 -- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 msgr2=0x7f0a60062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:31.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.091+0000 7f0a81f6a700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 0x7f0a60062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:31.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.095+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0a7404da20 con 0x7f0a7c158c40 2023-10-06T11:23:31.100 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:31.095+0000 7f3d1dacf000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:23:31.137 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.130+0000 7f6932842000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:23:31.168 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:31.163+0000 7f3d1dacf000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:23:31.233 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:31.227+0000 7f3d1dacf000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:23:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.275+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a7c06a3b0 con 0x7f0a7c158c40 2023-10-06T11:23:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.275+0000 7f0a71ffb700 1 -- 172.21.15.7:0/4155010451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17341 (secure 0 0 0) 0x7f0a7401a020 con 0x7f0a7c158c40 2023-10-06T11:23:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:31.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:16.399561+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/2634621881":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3120583769":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/330210624":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3694782510":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6833/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6832/33317":"2023-10-07T11:23:16.399486+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 msgr2=0x7f0a60062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 0x7f0a60062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 msgr2=0x7f0a7c0ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400cf60 tx=0x7f0a74007870 comp rx=0 tx=0).stop 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 shutdown_connections 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a60060700 0x7f0a60062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a7c158c40 0x7f0a7c0ad830 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7c0ca4c0 0x7f0a7c0afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 --2- 172.21.15.7:0/4155010451 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a7c0c8090 0x7f0a7c155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 >> 172.21.15.7:0/4155010451 conn(0x7f0a7c06bd70 msgr2=0x7f0a7c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 shutdown_connections 2023-10-06T11:23:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.279+0000 7f0a82f6c700 1 -- 172.21.15.7:0/4155010451 wait complete. 2023-10-06T11:23:31.292 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.286+0000 7f6932842000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:23:31.293 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:23:31.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:31.355 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.350+0000 7f6932842000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:23:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.387+0000 7f69213f2700 1 Processor -- start 2023-10-06T11:23:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.387+0000 7f69213f2700 1 -- start start 2023-10-06T11:23:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f691c0c9bb0 0x7f691c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c067e30 0x7f691c0682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0cc9f0 con 0x7f691c0c9f80 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c159460 con 0x7f691c0c9bb0 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0687f0 con 0x7f691c067e30 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f691c0c9bb0 0x7f691c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f691c0c9bb0 0x7f691c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49846/0 (socket says 172.21.15.7:49846) 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 -- 172.21.15.7:0/3069107264 learned_addr learned my addr 172.21.15.7:0/3069107264 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f691c0c9f80 0x7f691c0cb030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36548/0 (socket says 172.21.15.7:36548) 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691b7fe700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c067e30 0x7f691c0682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 -- 172.21.15.7:0/3069107264 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c067e30 msgr2=0x7f691c0682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c067e30 0x7f691c0682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 -- 172.21.15.7:0/3069107264 >> v1:172.21.15.7:6789/0 conn(0x7f691c0c9f80 legacy=0x7f691c0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 -- 172.21.15.7:0/3069107264 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0cd240 con 0x7f691c0c9bb0 2023-10-06T11:23:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f691affd700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c0c9bb0 0x7f691c0cd6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f690c0099b0 tx=0x7f690c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0b6e4b00c204e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f6919ffb700 1 -- 172.21.15.7:0/3069107264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00b860 con 0x7f691c0c9bb0 2023-10-06T11:23:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f6919ffb700 1 -- 172.21.15.7:0/3069107264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00b9c0 con 0x7f691c0c9bb0 2023-10-06T11:23:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f6919ffb700 1 -- 172.21.15.7:0/3069107264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00bd60 con 0x7f691c0c9bb0 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- 172.21.15.7:0/3069107264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c0c9bb0 msgr2=0x7f691c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c0c9bb0 0x7f691c0cd6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f690c0099b0 tx=0x7f690c009250 comp rx=0 tx=0).stop 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- 172.21.15.7:0/3069107264 shutdown_connections 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c067e30 0x7f691c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 --2- 172.21.15.7:0/3069107264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c0c9bb0 0x7f691c0cd6d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- 172.21.15.7:0/3069107264 >> 172.21.15.7:0/3069107264 conn(0x7f691c06b8a0 msgr2=0x7f691c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- 172.21.15.7:0/3069107264 shutdown_connections 2023-10-06T11:23:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.391+0000 7f69213f2700 1 -- 172.21.15.7:0/3069107264 wait complete. 2023-10-06T11:23:31.398 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f3d1dacf000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:23:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 Processor -- start 2023-10-06T11:23:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 -- start start 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 0x7f691c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 0x7f691c1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c15b620 con 0x7f691c067e30 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0b1d80 con 0x7f691c0c9bb0 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f69213f2700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c14ff60 con 0x7f691c0c9f80 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f691affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f691affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49850/0 (socket says 172.21.15.7:49850) 2023-10-06T11:23:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.395+0000 7f691affd700 1 -- 172.21.15.7:0/1619585702 learned_addr learned my addr 172.21.15.7:0/1619585702 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691a7fc700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 0x7f691c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691b7fe700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 0x7f691c1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 msgr2=0x7f691c1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 0x7f691c1569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 msgr2=0x7f691c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 0x7f691c14fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 -- 172.21.15.7:0/1619585702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0094f0 con 0x7f691c067e30 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691b7fe700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 0x7f691c1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691a7fc700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 0x7f691c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691affd700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f690c009fd0 tx=0x7f690c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c024440 con 0x7f691c067e30 2023-10-06T11:23:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c0245a0 con 0x7f691c067e30 2023-10-06T11:23:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c0b1170 con 0x7f691c067e30 2023-10-06T11:23:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c024940 con 0x7f691c067e30 2023-10-06T11:23:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c0b1730 con 0x7f691c067e30 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c04be00 con 0x7f691c067e30 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f690c01a420 con 0x7f691c067e30 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 0x7f6908062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691a7fc700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 msgr2=0x7f6908062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f690c07ed70 con 0x7f691c067e30 2023-10-06T11:23:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.399+0000 7f691a7fc700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 0x7f6908062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.407+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f690c04ed40 con 0x7f691c067e30 2023-10-06T11:23:31.486 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:31.479+0000 7f3d1dacf000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:23:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.603+0000 7f691a7fc700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 msgr2=0x7f6908062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.603+0000 7f691a7fc700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 0x7f6908062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:23:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.615+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f691c0cd770 con 0x7f691c067e30 2023-10-06T11:23:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.615+0000 7f6902ffd700 1 -- 172.21.15.7:0/1619585702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7f690c015020 con 0x7f691c067e30 2023-10-06T11:23:31.618 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:31.618 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:31.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 msgr2=0x7f6908062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:31.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 0x7f6908062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 msgr2=0x7f691c15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f690c009fd0 tx=0x7f690c017000 comp rx=0 tx=0).stop 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 shutdown_connections 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6908060700 0x7f6908062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f691c0c9f80 0x7f691c1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f691c0c9bb0 0x7f691c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 --2- 172.21.15.7:0/1619585702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f691c067e30 0x7f691c15b0e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 >> 172.21.15.7:0/1619585702 conn(0x7f691c06b8a0 msgr2=0x7f691c0c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 shutdown_connections 2023-10-06T11:23:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.619+0000 7f69213f2700 1 -- 172.21.15.7:0/1619585702 wait complete. 2023-10-06T11:23:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:31.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:23:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 Processor -- start 2023-10-06T11:23:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 -- start start 2023-10-06T11:23:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ac406eb80 0x7f8ac406bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac406e6f0 con 0x7f8ac4155850 2023-10-06T11:23:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40cbc70 con 0x7f8ac406bad0 2023-10-06T11:23:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8aca480700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40cbf50 con 0x7f8ac406eb80 2023-10-06T11:23:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8ac9c7f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ac4155850 0x7f8ac4155c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36556/0 (socket says 172.21.15.7:36556) 2023-10-06T11:23:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8ac9c7f700 1 -- 172.21.15.7:0/1684913588 learned_addr learned my addr 172.21.15.7:0/1684913588 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8ac8c7d700 1 --2- 172.21.15.7:0/1684913588 >> v2:172.21.15.50:3300/0 conn(0x7f8ac406eb80 0x7f8ac406bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842592244 0 0) 0x7f8ac406e6f0 con 0x7f8ac4155850 2023-10-06T11:23:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.739+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab0003570 con 0x7f8ac4155850 2023-10-06T11:23:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1327099076 0 0) 0x7f8ab0003570 con 0x7f8ac4155850 2023-10-06T11:23:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 >> v2:172.21.15.50:3300/0 conn(0x7f8ac406eb80 msgr2=0x7f8ac406bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 --2- 172.21.15.7:0/1684913588 >> v2:172.21.15.50:3300/0 conn(0x7f8ac406eb80 0x7f8ac406bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 >> v1:172.21.15.50:6789/0 conn(0x7f8ac406bad0 legacy=0x7f8ac40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac406e0a0 con 0x7f8ac4155850 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4242410576 0 0) 0x7f8abc0017f0 con 0x7f8ac4155850 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 >> v1:172.21.15.7:6789/0 conn(0x7f8ac4155850 legacy=0x7f8ac4155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ab0003a60 0x7f8ab0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab0004810 0x7f8ab0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 0x7f8ab000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40cbf50 con 0x7f8ab00051c0 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000e2e0 con 0x7f8ab0004810 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000e4a0 con 0x7f8ab0003a60 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 0x7f8ab000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac947e700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ab0003a60 0x7f8ab0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac8c7d700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab0004810 0x7f8ab0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 -- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ab0003a60 msgr2=0x7f8ab0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ab0003a60 0x7f8ab0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 -- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab0004810 msgr2=0x7f8ab0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab0004810 0x7f8ab0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 -- 172.21.15.7:0/1684913588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0003570 con 0x7f8ab00051c0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 -- 172.21.15.7:0/1684913588 reap_dead start 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8ac9c7f700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 0x7f8ab000dc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00a930 tx=0x7f8abc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319a8cd6ecbf7154 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0070d0 con 0x7f8ab00051c0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc007250 con 0x7f8ab00051c0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8abbfff700 1 -- 172.21.15.7:0/1684913588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc007610 con 0x7f8ab00051c0 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 -- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 msgr2=0x7f8ab000dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 0x7f8ab000dc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00a930 tx=0x7f8abc0021d0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 -- 172.21.15.7:0/1684913588 shutdown_connections 2023-10-06T11:23:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 --2- 172.21.15.7:0/1684913588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ab00051c0 0x7f8ab000dc70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 -- 172.21.15.7:0/1684913588 >> 172.21.15.7:0/1684913588 conn(0x7f8ac40cc190 msgr2=0x7f8ac40cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 -- 172.21.15.7:0/1684913588 shutdown_connections 2023-10-06T11:23:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.743+0000 7f8aca480700 1 -- 172.21.15.7:0/1684913588 wait complete. 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 Processor -- start 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- start start 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab00051c0 0x7f8ac40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ac40b1d80 0x7f8ac414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40c5af0 con 0x7f8ac41696d0 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40c5570 con 0x7f8ab00051c0 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40c5810 con 0x7f8ac40b1d80 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49868/0 (socket says 172.21.15.7:49868) 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 -- 172.21.15.7:0/472169228 learned_addr learned my addr 172.21.15.7:0/472169228 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac9c7f700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ac40b1d80 0x7f8ac414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac947e700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab00051c0 0x7f8ac40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ac40b1d80 msgr2=0x7f8ac414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ac40b1d80 0x7f8ac414eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab00051c0 msgr2=0x7f8ac40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab00051c0 0x7f8ac40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 -- 172.21.15.7:0/472169228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4155dc0 con 0x7f8ac41696d0 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ac8c7d700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4011750 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4015670 con 0x7f8ac41696d0 2023-10-06T11:23:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab40157d0 con 0x7f8ac41696d0 2023-10-06T11:23:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac41560a0 con 0x7f8ac41696d0 2023-10-06T11:23:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4015b70 con 0x7f8ac41696d0 2023-10-06T11:23:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.747+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac414f1e0 con 0x7f8ac41696d0 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4155c20 con 0x7f8ac41696d0 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 10) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f8ab4013410 con 0x7f8ac41696d0 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8ab97fa700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 0x7f8aa8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8ac947e700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 msgr2=0x7f8aa8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f8ab407d140 con 0x7f8ac41696d0 2023-10-06T11:23:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.751+0000 7f8ac947e700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 0x7f8aa8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:23:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.759+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ac4155c20 con 0x7f8ac41696d0 2023-10-06T11:23:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.955+0000 7f8ac947e700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 msgr2=0x7f8aa8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6832/33317 2023-10-06T11:23:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.955+0000 7f8ac947e700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 0x7f8aa8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:23:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.955+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ac4155c20 con 0x7f8ac41696d0 2023-10-06T11:23:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.955+0000 7f8ab97fa700 1 -- 172.21.15.7:0/472169228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4585 (secure 0 0 0) 0x7f8ac4155c20 con 0x7f8ac41696d0 2023-10-06T11:23:31.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:31.960 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:23:02.931201+0000","modified":"2023-10-06T11:23:06.573482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3888808928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3888808928},{"type":"v1","addr":"172.21.15.7:6835","nonce":3888808928}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 msgr2=0x7f8aa8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 0x7f8aa8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 msgr2=0x7f8ac40b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4011750 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 shutdown_connections 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f8aa8060700 0x7f8aa8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f8ac40b1d80 0x7f8ac414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac41696d0 0x7f8ac40b1840 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 --2- 172.21.15.7:0/472169228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ab00051c0 0x7f8ac40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 >> 172.21.15.7:0/472169228 conn(0x7f8ac40cc190 msgr2=0x7f8ac40c2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 shutdown_connections 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:31.959+0000 7f8aca480700 1 -- 172.21.15.7:0/472169228 wait complete. 2023-10-06T11:23:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-10-06T11:23:31.971 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:31.966+0000 7f6932842000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:23:31.978 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:32.072 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68fca80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.072 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68fca80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68ffa86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68ffa86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68ffa86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68ffa86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68ffa86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68fca80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68fca80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.073 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-10-06T11:23:32.066+0000 7f68fca80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:23:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 Processor -- start 2023-10-06T11:23:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- start start 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 0x7fad10157d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad1006a3e0 0x7fad1006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad100abd70 con 0x7fad100c71e0 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1006c090 con 0x7fad1006a3e0 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100c6dd0 con 0x7fad10157970 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad1006a3e0 0x7fad1006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 0x7fad10157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0dd9b700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fad100c71e0 0x7fad10069cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:46058/0 (socket says 172.21.15.7:46058) 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 0x7fad10157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51934/0 (socket says 172.21.15.7:51934) 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad1006a3e0 0x7fad1006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35284/0 (socket says 172.21.15.7:35284) 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0dd9b700 1 -- 172.21.15.7:0/39556878 learned_addr learned my addr 172.21.15.7:0/39556878 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 -- 172.21.15.7:0/39556878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 msgr2=0x7fad10157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 --2- 172.21.15.7:0/39556878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 0x7fad10157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 -- 172.21.15.7:0/39556878 >> v1:172.21.15.50:6789/0 conn(0x7fad100c71e0 legacy=0x7fad10069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 -- 172.21.15.7:0/39556878 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100c7670 con 0x7fad1006a3e0 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0d59a700 1 -- 172.21.15.7:0/39556878 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425090542 0 0) 0x7fad100abd70 con 0x7fad100c71e0 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0ed9d700 1 --2- 172.21.15.7:0/39556878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1006a3e0 0x7fad1006e000 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad08009e60 tx=0x7fad08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62937433f344c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0d59a700 1 -- 172.21.15.7:0/39556878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800e070 con 0x7fad1006a3e0 2023-10-06T11:23:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0d59a700 1 -- 172.21.15.7:0/39556878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800bbf0 con 0x7fad1006a3e0 2023-10-06T11:23:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0d59a700 1 -- 172.21.15.7:0/39556878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08012430 con 0x7fad1006a3e0 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- 172.21.15.7:0/39556878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1006a3e0 msgr2=0x7fad1006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 --2- 172.21.15.7:0/39556878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1006a3e0 0x7fad1006e000 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad08009e60 tx=0x7fad08009780 comp rx=0 tx=0).stop 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- 172.21.15.7:0/39556878 shutdown_connections 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 --2- 172.21.15.7:0/39556878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad1006a3e0 0x7fad1006e000 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 --2- 172.21.15.7:0/39556878 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad10157970 0x7fad10157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- 172.21.15.7:0/39556878 >> 172.21.15.7:0/39556878 conn(0x7fad100cc190 msgr2=0x7fad100cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- 172.21.15.7:0/39556878 shutdown_connections 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.079+0000 7fad0f59e700 1 -- 172.21.15.7:0/39556878 wait complete. 2023-10-06T11:23:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 Processor -- start 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 -- start start 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 0x7fad10168550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 0x7fad1015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 0x7fad1015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b18a0 con 0x7fad10157970 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10168a90 con 0x7fad100c71e0 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0f59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b0c90 con 0x7fad1006a3e0 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 0x7fad10168550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 0x7fad1015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0e59c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 0x7fad10168550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51942/0 (socket says 172.21.15.7:51942) 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0dd9b700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 0x7fad1015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0e59c700 1 -- 172.21.15.7:0/4273849802 learned_addr learned my addr 172.21.15.7:0/4273849802 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 -- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 msgr2=0x7fad10168550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 0x7fad10168550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 -- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 msgr2=0x7fad1015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 0x7fad1015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 -- 172.21.15.7:0/4273849802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08009a20 con 0x7fad10157970 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0dd9b700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 0x7fad1015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.083+0000 7fad0ed9d700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 0x7fad1015d730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad08016000 tx=0x7fad08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08027470 con 0x7fad10157970 2023-10-06T11:23:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080275f0 con 0x7fad10157970 2023-10-06T11:23:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad100b0f70 con 0x7fad10157970 2023-10-06T11:23:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad080279b0 con 0x7fad10157970 2023-10-06T11:23:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad100b1530 con 0x7fad10157970 2023-10-06T11:23:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1006d240 con 0x7fad10157970 2023-10-06T11:23:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fad08024070 con 0x7fad10157970 2023-10-06T11:23:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.087+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6937+0+0 (secure 0 0 0) 0x7fad0807ee10 con 0x7fad10157970 2023-10-06T11:23:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.095+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fad0804f370 con 0x7fad10157970 2023-10-06T11:23:32.165 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-06T11:23:32.163+0000 7f3d1dacf000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:23:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.255+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad100b19e0 con 0x7fad10157970 2023-10-06T11:23:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7facfe7fc700 1 -- 172.21.15.7:0/4273849802 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17774 (secure 0 0 0) 0x7fad08052990 con 0x7fad10157970 2023-10-06T11:23:32.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:32.262 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:31.992820+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:23:00.705486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T11:23:01.914689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/1291207339":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1549220530":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/897696649":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1588488060":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3983929207":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/2318534492":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3428455953":"2023-10-07T11:23:31.992769+0000","172.21.15.50:6832/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6833/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3694782510":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3120583769":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/330210624":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/2634621881":"2023-10-07T11:23:16.399486+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 msgr2=0x7fad1015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 0x7fad1015d730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad08016000 tx=0x7fad08019000 comp rx=0 tx=0).stop 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 shutdown_connections 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad10157970 0x7fad1015d730 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad100c71e0 0x7fad1015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 --2- 172.21.15.7:0/4273849802 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fad1006a3e0 0x7fad10168550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 >> 172.21.15.7:0/4273849802 conn(0x7fad100cc190 msgr2=0x7fad100add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 shutdown_connections 2023-10-06T11:23:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.259+0000 7fad0f59e700 1 -- 172.21.15.7:0/4273849802 wait complete. 2023-10-06T11:23:32.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:23:32.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 Processor -- start 2023-10-06T11:23:32.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 -- start start 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd210155fd0 0x7fd2101563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd210159000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2100c7b20 con 0x7fd2100ca190 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd210159f30 con 0x7fd210155fd0 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20ffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100c7530 con 0x7fd2100c9d60 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd210155fd0 0x7fd2101563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd210155fd0 0x7fd2101563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35300/0 (socket says 172.21.15.7:35300) 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20effd700 1 -- 172.21.15.7:0/1182656566 learned_addr learned my addr 172.21.15.7:0/1182656566 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:32.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.379+0000 7fd20e7fc700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd210159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20effd700 1 -- 172.21.15.7:0/1182656566 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 msgr2=0x7fd210159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20effd700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd210159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20effd700 1 -- 172.21.15.7:0/1182656566 >> v1:172.21.15.50:6789/0 conn(0x7fd2100ca190 legacy=0x7fd210069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20effd700 1 -- 172.21.15.7:0/1182656566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2101568e0 con 0x7fd210155fd0 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20e7fc700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd210159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:32.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20effd700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2101563a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2000099b0 tx=0x7fd200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ea2fac1419979c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20dffb700 1 -- 172.21.15.7:0/1182656566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20000b860 con 0x7fd210155fd0 2023-10-06T11:23:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20dffb700 1 -- 172.21.15.7:0/1182656566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20000b9c0 con 0x7fd210155fd0 2023-10-06T11:23:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20dffb700 1 -- 172.21.15.7:0/1182656566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20000bd60 con 0x7fd210155fd0 2023-10-06T11:23:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- 172.21.15.7:0/1182656566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 msgr2=0x7fd2101563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2101563a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2000099b0 tx=0x7fd200009250 comp rx=0 tx=0).stop 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- 172.21.15.7:0/1182656566 shutdown_connections 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd210159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1182656566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2101563a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- 172.21.15.7:0/1182656566 >> 172.21.15.7:0/1182656566 conn(0x7fd2100cc190 msgr2=0x7fd2100cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- 172.21.15.7:0/1182656566 shutdown_connections 2023-10-06T11:23:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- 172.21.15.7:0/1182656566 wait complete. 2023-10-06T11:23:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 Processor -- start 2023-10-06T11:23:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.383+0000 7fd20ffff700 1 -- start start 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd2100b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2100ca190 0x7fd2100b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21016ac10 con 0x7fd210155fd0 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21016b790 con 0x7fd2100ca190 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21016ae60 con 0x7fd2100c9d60 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35302/0 (socket says 172.21.15.7:35302) 2023-10-06T11:23:32.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 -- 172.21.15.7:0/1665657889 learned_addr learned my addr 172.21.15.7:0/1665657889 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20effd700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd2100b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20e7fc700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2100ca190 0x7fd2100b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 -- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 msgr2=0x7fd2100b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd2100b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 -- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2100ca190 msgr2=0x7fd2100b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2100ca190 0x7fd2100b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 -- 172.21.15.7:0/1665657889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000094f0 con 0x7fd210155fd0 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20f7fe700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c8e0 tx=0x7fd208001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208012630 con 0x7fd210155fd0 2023-10-06T11:23:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2080127b0 con 0x7fd210155fd0 2023-10-06T11:23:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21016b140 con 0x7fd210155fd0 2023-10-06T11:23:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208012b70 con 0x7fd210155fd0 2023-10-06T11:23:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21014ca60 con 0x7fd210155fd0 2023-10-06T11:23:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.387+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21016b900 con 0x7fd210155fd0 2023-10-06T11:23:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.391+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fd208003030 con 0x7fd210155fd0 2023-10-06T11:23:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.391+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6937+0+0 (secure 0 0 0) 0x7fd20807cd30 con 0x7fd210155fd0 2023-10-06T11:23:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.399+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd20804d290 con 0x7fd210155fd0 2023-10-06T11:23:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.595+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd210159970 con 0x7fd210155fd0 2023-10-06T11:23:32.598 INFO:tasks.ceph.mon.a.smithi007.stderr:2023-10-06T11:23:32.595+0000 7f67b417b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:23:32.607 INFO:tasks.ceph.mds.a.smithi007.stderr: -3680> 2023-10-06T11:23:07.523+0000 7f554cdf6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:23:32.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.603+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/1665657889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd2080508b0 con 0x7fd210155fd0 2023-10-06T11:23:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 msgr2=0x7fd2100c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c8e0 tx=0x7fd208001710 comp rx=0 tx=0).stop 2023-10-06T11:23:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 shutdown_connections 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210155fd0 0x7fd2100c5450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2100ca190 0x7fd2100b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 --2- 172.21.15.7:0/1665657889 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd2100c9d60 0x7fd2100b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 >> 172.21.15.7:0/1665657889 conn(0x7fd2100cc190 msgr2=0x7fd2100c7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 shutdown_connections 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.607+0000 7fd20ffff700 1 -- 172.21.15.7:0/1665657889 wait complete. 2023-10-06T11:23:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:23:32.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:23:32.633 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-06T11:23:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.723+0000 7f361759e700 1 Processor -- start 2023-10-06T11:23:32.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.723+0000 7f361759e700 1 -- start start 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36181550f0 0x7f36180ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 0x7f3618069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36180cae10 0x7f361806a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180c7af0 con 0x7f36180cae10 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180cb3e0 con 0x7f36181554c0 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36180cb6c0 con 0x7f36181550f0 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36180cae10 0x7f361806a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36180cae10 0x7f361806a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35334/0 (socket says 172.21.15.7:35334) 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 -- 172.21.15.7:0/276439837 learned_addr learned my addr 172.21.15.7:0/276439837 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361659c700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36181550f0 0x7f36180ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3615d9b700 1 --2- 172.21.15.7:0/276439837 >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 0x7f3618069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 -- 172.21.15.7:0/276439837 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36181550f0 msgr2=0x7f36180ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36181550f0 0x7f36180ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 -- 172.21.15.7:0/276439837 >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 msgr2=0x7f3618069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 --2- 172.21.15.7:0/276439837 >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 0x7f3618069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 -- 172.21.15.7:0/276439837 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180c7780 con 0x7f36180cae10 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3615d9b700 1 --2- 172.21.15.7:0/276439837 >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 0x7f3618069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f3616d9d700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180cae10 0x7f361806a210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f361806da10 tx=0x7f361000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eaf4918f12da6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361559a700 1 -- 172.21.15.7:0/276439837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000aa10 con 0x7f36180cae10 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361559a700 1 -- 172.21.15.7:0/276439837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361000ab70 con 0x7f36180cae10 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361559a700 1 -- 172.21.15.7:0/276439837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610011410 con 0x7f36180cae10 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 -- 172.21.15.7:0/276439837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180cae10 msgr2=0x7f361806a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.727+0000 7f361759e700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180cae10 0x7f361806a210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f361806da10 tx=0x7f361000b040 comp rx=0 tx=0).stop 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- 172.21.15.7:0/276439837 shutdown_connections 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180cae10 0x7f361806a210 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- 172.21.15.7:0/276439837 >> v2:172.21.15.50:3300/0 conn(0x7f36181554c0 0x7f3618069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- 172.21.15.7:0/276439837 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f36181550f0 0x7f36180ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- 172.21.15.7:0/276439837 >> 172.21.15.7:0/276439837 conn(0x7f3618067e30 msgr2=0x7f3618068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- 172.21.15.7:0/276439837 shutdown_connections 2023-10-06T11:23:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- 172.21.15.7:0/276439837 wait complete. 2023-10-06T11:23:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 Processor -- start 2023-10-06T11:23:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- start start 2023-10-06T11:23:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180cae10 0x7f36181505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3618152150 0x7f3618153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36180c4150 con 0x7f36181550f0 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181580e0 con 0x7f36180cae10 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3618150b30 con 0x7f3618152150 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f3615d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f3615d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35350/0 (socket says 172.21.15.7:35350) 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f3615d9b700 1 -- 172.21.15.7:0/2799201059 learned_addr learned my addr 172.21.15.7:0/2799201059 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f361659c700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180cae10 0x7f36181505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.731+0000 7f3616d9d700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3618152150 0x7f3618153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 -- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3618152150 msgr2=0x7f3618153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3618152150 0x7f3618153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 -- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180cae10 msgr2=0x7f36181505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180cae10 0x7f36181505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 -- 172.21.15.7:0/2799201059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009a40 con 0x7f36181550f0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f3615d9b700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f360c00d860 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0125f0 con 0x7f36181550f0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c012750 con 0x7f36181550f0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618157230 con 0x7f36181550f0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c012af0 con 0x7f36181550f0 2023-10-06T11:23:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3618157580 con 0x7f36181550f0 2023-10-06T11:23:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36180c4680 con 0x7f36181550f0 2023-10-06T11:23:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.735+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f360c013480 con 0x7f36181550f0 2023-10-06T11:23:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.739+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f360c07c970 con 0x7f36181550f0 2023-10-06T11:23:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.743+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f36180c4680 con 0x7f36181550f0 2023-10-06T11:23:32.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:32.935+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f36180c4680 con 0x7f36181550f0 2023-10-06T11:23:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.007+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f360c04c8b0 con 0x7f36181550f0 2023-10-06T11:23:33.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.007+0000 7f36067fc700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 0x7f35f8068d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.007+0000 7f361659c700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 0x7f35f8068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.011+0000 7f361659c700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 0x7f35f8068d50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3608015490 tx=0x7f360800f040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:33.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.603+0000 7f36067fc700 1 -- 172.21.15.7:0/2799201059 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f360c068db0 con 0x7f36181550f0 2023-10-06T11:23:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 msgr2=0x7f35f8068d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 0x7f35f8068d50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3608015490 tx=0x7f360800f040 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 msgr2=0x7f3618151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f360c00d860 tx=0x7f360c008040 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 shutdown_connections 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f35f8066970 0x7f35f8068d50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3618152150 0x7f3618153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181550f0 0x7f3618151c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 --2- 172.21.15.7:0/2799201059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180cae10 0x7f36181505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 >> 172.21.15.7:0/2799201059 conn(0x7f3618067e30 msgr2=0x7f36180b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 shutdown_connections 2023-10-06T11:23:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.611+0000 7f361759e700 1 -- 172.21.15.7:0/2799201059 wait complete. 2023-10-06T11:23:33.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:23:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.751+0000 7f2d8759e700 1 Processor -- start 2023-10-06T11:23:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 -- start start 2023-10-06T11:23:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 0x7f2d88066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d8806f090 0x7f2d881593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8815cbc0 con 0x7f2d8806e070 2023-10-06T11:23:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d881598f0 con 0x7f2d8806f090 2023-10-06T11:23:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8759e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d88159b90 con 0x7f2d8806ecc0 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d8806f090 0x7f2d881593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d8806f090 0x7f2d881593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35358/0 (socket says 172.21.15.7:35358) 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 -- 172.21.15.7:0/345645903 learned_addr learned my addr 172.21.15.7:0/345645903 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8659c700 1 --2- 172.21.15.7:0/345645903 >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 0x7f2d88066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 -- 172.21.15.7:0/345645903 >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 msgr2=0x7f2d88066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 --2- 172.21.15.7:0/345645903 >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 0x7f2d88066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 -- 172.21.15.7:0/345645903 >> v1:172.21.15.50:6789/0 conn(0x7f2d8806e070 legacy=0x7f2d8815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 -- 172.21.15.7:0/345645903 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d88159e70 con 0x7f2d8806f090 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d8659c700 1 --2- 172.21.15.7:0/345645903 >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 0x7f2d88066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d7ffff700 1 --2- 172.21.15.7:0/345645903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806f090 0x7f2d881593b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806d950 tx=0x7f2d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f57c8da5b30e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.755+0000 7f2d85d9b700 1 -- 172.21.15.7:0/345645903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000b860 con 0x7f2d8806f090 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d85d9b700 1 -- 172.21.15.7:0/345645903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000b9c0 con 0x7f2d8806f090 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d85d9b700 1 -- 172.21.15.7:0/345645903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bd60 con 0x7f2d8806f090 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 -- 172.21.15.7:0/345645903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806f090 msgr2=0x7f2d881593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 --2- 172.21.15.7:0/345645903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806f090 0x7f2d881593b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806d950 tx=0x7f2d70009250 comp rx=0 tx=0).stop 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 -- 172.21.15.7:0/345645903 shutdown_connections 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 --2- 172.21.15.7:0/345645903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806f090 0x7f2d881593b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 --2- 172.21.15.7:0/345645903 >> v2:172.21.15.50:3300/0 conn(0x7f2d8806ecc0 0x7f2d88066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 -- 172.21.15.7:0/345645903 >> 172.21.15.7:0/345645903 conn(0x7f2d880cc190 msgr2=0x7f2d880cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 -- 172.21.15.7:0/345645903 shutdown_connections 2023-10-06T11:23:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.759+0000 7f2d8759e700 1 -- 172.21.15.7:0/345645903 wait complete. 2023-10-06T11:23:33.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 Processor -- start 2023-10-06T11:23:33.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- start start 2023-10-06T11:23:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2d8806ecc0 0x7f2d88140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8806f090 0x7f2d880aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88141d40 con 0x7f2d8806e070 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88141130 con 0x7f2d8806f090 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88141410 con 0x7f2d8806ecc0 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35372/0 (socket says 172.21.15.7:35372) 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 -- 172.21.15.7:0/2577642731 learned_addr learned my addr 172.21.15.7:0/2577642731 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d7ffff700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2d8806ecc0 0x7f2d88140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d86d9d700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8806f090 0x7f2d880aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 -- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2d8806ecc0 msgr2=0x7f2d88140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2d8806ecc0 0x7f2d88140900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 -- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8806f090 msgr2=0x7f2d880aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8806f090 0x7f2d880aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 -- 172.21.15.7:0/2577642731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094f0 con 0x7f2d8806e070 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8659c700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e920 tx=0x7f2d78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78005a00 con 0x7f2d8806e070 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d78005b60 con 0x7f2d8806e070 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d881416f0 con 0x7f2d8806e070 2023-10-06T11:23:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78006020 con 0x7f2d8806e070 2023-10-06T11:23:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.763+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d88140e40 con 0x7f2d8806e070 2023-10-06T11:23:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8806cfd0 con 0x7f2d8806e070 2023-10-06T11:23:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f2d780063d0 con 0x7f2d8806e070 2023-10-06T11:23:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d7e7fc700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 0x7f2d68062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f2d7807c9b0 con 0x7f2d8806e070 2023-10-06T11:23:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d7ffff700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 0x7f2d68062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.767+0000 7f2d7ffff700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 0x7f2d68062850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806d950 tx=0x7f2d70017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:33.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.775+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2d7804cde0 con 0x7f2d8806e070 2023-10-06T11:23:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:33.943+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d8804b5c0 con 0x7f2d8806e070 2023-10-06T11:23:34.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.615+0000 7f2d7e7fc700 1 -- 172.21.15.7:0/2577642731 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2d78050400 con 0x7f2d8806e070 2023-10-06T11:23:34.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 msgr2=0x7f2d68062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 0x7f2d68062850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806d950 tx=0x7f2d70017000 comp rx=0 tx=0).stop 2023-10-06T11:23:34.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 msgr2=0x7f2d881403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e920 tx=0x7f2d78008040 comp rx=0 tx=0).stop 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 shutdown_connections 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2d680603a0 0x7f2d68062850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8806f090 0x7f2d880aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2d8806ecc0 0x7f2d88140900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 --2- 172.21.15.7:0/2577642731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d8806e070 0x7f2d881403c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 >> 172.21.15.7:0/2577642731 conn(0x7f2d880cc190 msgr2=0x7f2d88069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.619+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 shutdown_connections 2023-10-06T11:23:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.623+0000 7f2d8759e700 1 -- 172.21.15.7:0/2577642731 wait complete. 2023-10-06T11:23:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:23:34.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:23:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.743+0000 7f9a01d2e700 1 Processor -- start 2023-10-06T11:23:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.743+0000 7f9a01d2e700 1 -- start start 2023-10-06T11:23:34.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a01d2e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f99fc0ccd70 0x7f99fc0ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a01d2e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0cac70 con 0x7f99fc158ea0 2023-10-06T11:23:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a01d2e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0caea0 con 0x7f99fc06d940 2023-10-06T11:23:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a01d2e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc068d40 con 0x7f99fc0ccd70 2023-10-06T11:23:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a00d2c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99fc158ea0 0x7f99fc0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43012/0 (socket says 172.21.15.7:43012) 2023-10-06T11:23:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f9a00d2c700 1 -- 172.21.15.7:0/402046948 learned_addr learned my addr 172.21.15.7:0/402046948 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fbfff700 1 --2- 172.21.15.7:0/402046948 >> v2:172.21.15.50:3300/0 conn(0x7f99fc0ccd70 0x7f99fc0ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230401747 0 0) 0x7f99fc0cac70 con 0x7f99fc158ea0 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e8003570 con 0x7f99fc158ea0 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561605948 0 0) 0x7f99fc0caea0 con 0x7f99fc06d940 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99fc0cac70 con 0x7f99fc06d940 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fbfff700 1 -- 172.21.15.7:0/402046948 >> v1:172.21.15.50:6789/0 conn(0x7f99fc06d940 legacy=0x7f99fc0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fbfff700 1 -- 172.21.15.7:0/402046948 >> v1:172.21.15.7:6789/0 conn(0x7f99fc158ea0 legacy=0x7f99fc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fbfff700 1 -- 172.21.15.7:0/402046948 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0cb180 con 0x7f99fc0ccd70 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fbfff700 1 --2- 172.21.15.7:0/402046948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc0ccd70 0x7f99fc0ca730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99fc068e80 tx=0x7f99ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a7d2cd6b65d9b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011630 con 0x7f99fc0ccd70 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec011790 con 0x7f99fc0ccd70 2023-10-06T11:23:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.747+0000 7f99fb7fe700 1 -- 172.21.15.7:0/402046948 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011b10 con 0x7f99fc0ccd70 2023-10-06T11:23:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- 172.21.15.7:0/402046948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc0ccd70 msgr2=0x7f99fc0ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/402046948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc0ccd70 0x7f99fc0ca730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99fc068e80 tx=0x7f99ec0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- 172.21.15.7:0/402046948 shutdown_connections 2023-10-06T11:23:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/402046948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc0ccd70 0x7f99fc0ca730 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- 172.21.15.7:0/402046948 >> 172.21.15.7:0/402046948 conn(0x7f99fc067e30 msgr2=0x7f99fc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- 172.21.15.7:0/402046948 shutdown_connections 2023-10-06T11:23:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- 172.21.15.7:0/402046948 wait complete. 2023-10-06T11:23:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 Processor -- start 2023-10-06T11:23:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.751+0000 7f9a01d2e700 1 -- start start 2023-10-06T11:23:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 0x7f99fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 0x7f99fc15a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15dab0 con 0x7f99fc06d940 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15cea0 con 0x7f99fc158ea0 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15d180 con 0x7f99fc0ccd70 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35374/0 (socket says 172.21.15.7:35374) 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 -- 172.21.15.7:0/2512253343 learned_addr learned my addr 172.21.15.7:0/2512253343 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a0152d700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 0x7f99fc15a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f99fbfff700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 0x7f99fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 -- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 msgr2=0x7f99fc0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 0x7f99fc0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 -- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 msgr2=0x7f99fc15a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 0x7f99fc15a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 -- 172.21.15.7:0/2512253343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009420 con 0x7f99fc06d940 2023-10-06T11:23:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f99fbfff700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 0x7f99fc0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a0152d700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 0x7f99fc15a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a00d2c700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f99e40058d0 tx=0x7f99e4007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4005b30 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e4005cb0 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc15d460 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4007320 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.755+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc15cb70 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc06c5c0 con 0x7f99fc06d940 2023-10-06T11:23:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f99e4007690 con 0x7f99fc06d940 2023-10-06T11:23:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f99f8ff9700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 0x7f99dc062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f99e407bc50 con 0x7f99fc06d940 2023-10-06T11:23:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f99fbfff700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 0x7f99dc062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.759+0000 7f99fbfff700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 0x7f99dc062880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99fc068e80 tx=0x7f99ec016000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.767+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99e404c180 con 0x7f99fc06d940 2023-10-06T11:23:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:34.927+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99fc1531e0 con 0x7f99fc06d940 2023-10-06T11:23:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.015+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f99e40550d0 con 0x7f99fc06d940 2023-10-06T11:23:35.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:35.199 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:35.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.615+0000 7f99f8ff9700 1 -- 172.21.15.7:0/2512253343 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99e404f980 con 0x7f99fc06d940 2023-10-06T11:23:35.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.619+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 msgr2=0x7f99dc062880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.619+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 0x7f99dc062880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99fc068e80 tx=0x7f99ec016000 comp rx=0 tx=0).stop 2023-10-06T11:23:35.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.619+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 msgr2=0x7f99fc152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.619+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f99e40058d0 tx=0x7f99e4007e50 comp rx=0 tx=0).stop 2023-10-06T11:23:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 shutdown_connections 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f99dc0603d0 0x7f99dc062880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99fc158ea0 0x7f99fc15a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f99fc0ccd70 0x7f99fc0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 --2- 172.21.15.7:0/2512253343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc06d940 0x7f99fc152160 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 >> 172.21.15.7:0/2512253343 conn(0x7f99fc067e30 msgr2=0x7f99fc0c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 shutdown_connections 2023-10-06T11:23:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.623+0000 7f9a01d2e700 1 -- 172.21.15.7:0/2512253343 wait complete. 2023-10-06T11:23:35.629 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-10-06T11:23:35.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:23:35.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.743+0000 7f02ed243700 1 Processor -- start 2023-10-06T11:23:35.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.743+0000 7f02ed243700 1 -- start start 2023-10-06T11:23:35.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.743+0000 7f02ed243700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e81569b0 0x7f02e806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.743+0000 7f02ed243700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f02e80c9240 0x7f02e815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e815b6e0 con 0x7f02e8156d80 2023-10-06T11:23:35.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e8066c00 con 0x7f02e81569b0 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e806e780 con 0x7f02e80c9240 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e81569b0 0x7f02e806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02df7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f02e8156d80 0x7f02e80c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43028/0 (socket says 172.21.15.7:43028) 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e81569b0 0x7f02e806ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35382/0 (socket says 172.21.15.7:35382) 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02df7fe700 1 -- 172.21.15.7:0/792872988 learned_addr learned my addr 172.21.15.7:0/792872988 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02eca42700 1 --2- 172.21.15.7:0/792872988 >> v2:172.21.15.50:3300/0 conn(0x7f02e80c9240 0x7f02e815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 -- 172.21.15.7:0/792872988 >> v2:172.21.15.50:3300/0 conn(0x7f02e80c9240 msgr2=0x7f02e815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 --2- 172.21.15.7:0/792872988 >> v2:172.21.15.50:3300/0 conn(0x7f02e80c9240 0x7f02e815a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 -- 172.21.15.7:0/792872988 >> v1:172.21.15.7:6789/0 conn(0x7f02e8156d80 legacy=0x7f02e80c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e77fe700 1 -- 172.21.15.7:0/792872988 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693586070 0 0) 0x7f02e815b6e0 con 0x7f02e8156d80 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 -- 172.21.15.7:0/792872988 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e815b400 con 0x7f02e81569b0 2023-10-06T11:23:35.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e7fff700 1 --2- 172.21.15.7:0/792872988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81569b0 0x7f02e806ec10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f02e80cd200 tx=0x7f02d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a3fc415f7cd9bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:35.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e77fe700 1 -- 172.21.15.7:0/792872988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011630 con 0x7f02e81569b0 2023-10-06T11:23:35.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e77fe700 1 -- 172.21.15.7:0/792872988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d8011790 con 0x7f02e81569b0 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02e77fe700 1 -- 172.21.15.7:0/792872988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011b30 con 0x7f02e81569b0 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- 172.21.15.7:0/792872988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81569b0 msgr2=0x7f02e806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 --2- 172.21.15.7:0/792872988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81569b0 0x7f02e806ec10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f02e80cd200 tx=0x7f02d80091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- 172.21.15.7:0/792872988 shutdown_connections 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 --2- 172.21.15.7:0/792872988 >> v2:172.21.15.50:3300/0 conn(0x7f02e80c9240 0x7f02e815a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 --2- 172.21.15.7:0/792872988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81569b0 0x7f02e806ec10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- 172.21.15.7:0/792872988 >> 172.21.15.7:0/792872988 conn(0x7f02e8067e30 msgr2=0x7f02e8068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- 172.21.15.7:0/792872988 shutdown_connections 2023-10-06T11:23:35.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.747+0000 7f02ed243700 1 -- 172.21.15.7:0/792872988 wait complete. 2023-10-06T11:23:35.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.751+0000 7f02ed243700 1 Processor -- start 2023-10-06T11:23:35.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.751+0000 7f02ed243700 1 -- start start 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02e81569b0 0x7f02e813eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e813f030 con 0x7f02e8156d80 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e814b850 con 0x7f02e81569b0 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e814c690 con 0x7f02e80c9240 2023-10-06T11:23:35.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:51994/0 (socket says 172.21.15.7:51994) 2023-10-06T11:23:35.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e7fff700 1 -- 172.21.15.7:0/1516384031 learned_addr learned my addr 172.21.15.7:0/1516384031 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:35.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35384/0 (socket says 172.21.15.7:35384) 2023-10-06T11:23:35.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02df7fe700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02e81569b0 0x7f02e813eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 -- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 msgr2=0x7f02e80b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 -- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02e81569b0 msgr2=0x7f02e813eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02e81569b0 0x7f02e813eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 -- 172.21.15.7:0/1516384031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8009420 con 0x7f02e8156d80 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e7fff700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02eca42700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009e50 tx=0x7f02e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000caa0 con 0x7f02e8156d80 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e000f040 con 0x7f02e8156d80 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000ce50 con 0x7f02e8156d80 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e814ba80 con 0x7f02e8156d80 2023-10-06T11:23:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.755+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e814c040 con 0x7f02e8156d80 2023-10-06T11:23:35.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e806cf60 con 0x7f02e8156d80 2023-10-06T11:23:35.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f02e0014690 con 0x7f02e8156d80 2023-10-06T11:23:35.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02e4ff9700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 0x7f02c8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:35.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f02e007d7f0 con 0x7f02e8156d80 2023-10-06T11:23:35.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02e7fff700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 0x7f02c8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:35.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.759+0000 7f02e7fff700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 0x7f02c8062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02d8011fd0 tx=0x7f02d8017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:35.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.771+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f02e00513e0 con 0x7f02e8156d80 2023-10-06T11:23:35.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.931+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f02e806e130 con 0x7f02e8156d80 2023-10-06T11:23:35.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.939+0000 7f02e4ff9700 1 -- 172.21.15.7:0/1516384031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7f02e0050f70 con 0x7f02e8156d80 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 msgr2=0x7f02c8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 0x7f02c8062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02d8011fd0 tx=0x7f02d8017000 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 msgr2=0x7f02e814b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009e50 tx=0x7f02e000b040 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 shutdown_connections 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f02c8060670 0x7f02c8062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8156d80 0x7f02e814b310 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02e81569b0 0x7f02e813eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 --2- 172.21.15.7:0/1516384031 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f02e80c9240 0x7f02e80b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 >> 172.21.15.7:0/1516384031 conn(0x7f02e8067e30 msgr2=0x7f02e8068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:35.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 shutdown_connections 2023-10-06T11:23:35.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:35.943+0000 7f02ed243700 1 -- 172.21.15.7:0/1516384031 wait complete. 2023-10-06T11:23:35.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:23:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 Processor -- start 2023-10-06T11:23:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 -- start start 2023-10-06T11:23:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4680b41e0 0x7ff4680b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff46806de10 0x7ff468159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680b1190 con 0x7ff4680b3e10 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680b0f80 con 0x7ff4680b41e0 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46eb5c700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680b1470 con 0x7ff46806de10 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4680b41e0 0x7ff4680b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4680b41e0 0x7ff4680b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35386/0 (socket says 172.21.15.7:35386) 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46db5a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4680b3e10 0x7ff46806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43038/0 (socket says 172.21.15.7:43038) 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 -- 172.21.15.7:0/2842093088 learned_addr learned my addr 172.21.15.7:0/2842093088 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46e35b700 1 --2- 172.21.15.7:0/2842093088 >> v2:172.21.15.50:3300/0 conn(0x7ff46806de10 0x7ff468159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 -- 172.21.15.7:0/2842093088 >> v2:172.21.15.50:3300/0 conn(0x7ff46806de10 msgr2=0x7ff468159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 --2- 172.21.15.7:0/2842093088 >> v2:172.21.15.50:3300/0 conn(0x7ff46806de10 0x7ff468159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 -- 172.21.15.7:0/2842093088 >> v1:172.21.15.7:6789/0 conn(0x7ff4680b3e10 legacy=0x7ff46806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 -- 172.21.15.7:0/2842093088 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680b1750 con 0x7ff4680b41e0 2023-10-06T11:23:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46cb58700 1 -- 172.21.15.7:0/2842093088 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237786187 0 0) 0x7ff4680b1190 con 0x7ff4680b3e10 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46d359700 1 --2- 172.21.15.7:0/2842093088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b41e0 0x7ff4680b1f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff468159560 tx=0x7ff458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f269cd7836ca284 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46cb58700 1 -- 172.21.15.7:0/2842093088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800b860 con 0x7ff4680b41e0 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46cb58700 1 -- 172.21.15.7:0/2842093088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800b9c0 con 0x7ff4680b41e0 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.071+0000 7ff46cb58700 1 -- 172.21.15.7:0/2842093088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800bd60 con 0x7ff4680b41e0 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/2842093088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b41e0 msgr2=0x7ff4680b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/2842093088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b41e0 0x7ff4680b1f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff468159560 tx=0x7ff458009250 comp rx=0 tx=0).stop 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/2842093088 shutdown_connections 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/2842093088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b41e0 0x7ff4680b1f50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/2842093088 >> v2:172.21.15.50:3300/0 conn(0x7ff46806de10 0x7ff468159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/2842093088 >> 172.21.15.7:0/2842093088 conn(0x7ff4680cfd00 msgr2=0x7ff4680d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/2842093088 shutdown_connections 2023-10-06T11:23:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/2842093088 wait complete. 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 Processor -- start 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- start start 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 0x7ff46816cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff4680b41e0 0x7ff468153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46816db10 con 0x7ff4680b3e10 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46816d060 con 0x7ff46806de10 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681608f0 con 0x7ff4680b41e0 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35398/0 (socket says 172.21.15.7:35398) 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 -- 172.21.15.7:0/857389421 learned_addr learned my addr 172.21.15.7:0/857389421 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46e35b700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff4680b41e0 0x7ff468153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46db5a700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 0x7ff46816cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 -- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff4680b41e0 msgr2=0x7ff468153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff4680b41e0 0x7ff468153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 -- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 msgr2=0x7ff46816cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 0x7ff46816cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 -- 172.21.15.7:0/857389421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580094f0 con 0x7ff4680b3e10 2023-10-06T11:23:36.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46db5a700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 0x7ff46816cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:36.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46d359700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff458006f50 tx=0x7ff458017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458024440 con 0x7ff4680b3e10 2023-10-06T11:23:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4580245e0 con 0x7ff4680b3e10 2023-10-06T11:23:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4681694e0 con 0x7ff4680b3e10 2023-10-06T11:23:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580249a0 con 0x7ff4680b3e10 2023-10-06T11:23:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.075+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46816d830 con 0x7ff4680b3e10 2023-10-06T11:23:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.079+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46806c940 con 0x7ff4680b3e10 2023-10-06T11:23:36.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.079+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff458024d50 con 0x7ff4680b3e10 2023-10-06T11:23:36.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.079+0000 7ff45dffb700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 0x7ff44c062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.083+0000 7ff46db5a700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 0x7ff44c062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.083+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff45807ecd0 con 0x7ff4680b3e10 2023-10-06T11:23:36.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.083+0000 7ff46db5a700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 0x7ff44c062bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4640011e0 tx=0x7ff464007c90 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.087+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff45804efc0 con 0x7ff4680b3e10 2023-10-06T11:23:36.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.247+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff4680b1a30 con 0x7ff4680b3e10 2023-10-06T11:23:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff45dffb700 1 -- 172.21.15.7:0/857389421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff4580525e0 con 0x7ff4680b3e10 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 msgr2=0x7ff44c062bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 0x7ff44c062bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4640011e0 tx=0x7ff464007c90 comp rx=0 tx=0).stop 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 msgr2=0x7ff46814e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff458006f50 tx=0x7ff458017000 comp rx=0 tx=0).stop 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 shutdown_connections 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff44c060720 0x7ff44c062bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff4680b41e0 0x7ff468153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4680b3e10 0x7ff46814e8f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 --2- 172.21.15.7:0/857389421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff46806de10 0x7ff46816cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 >> 172.21.15.7:0/857389421 conn(0x7ff4680cfd00 msgr2=0x7ff468157940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 shutdown_connections 2023-10-06T11:23:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.259+0000 7ff46eb5c700 1 -- 172.21.15.7:0/857389421 wait complete. 2023-10-06T11:23:36.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:23:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.383+0000 7f4499138700 1 Processor -- start 2023-10-06T11:23:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.383+0000 7f4499138700 1 -- start start 2023-10-06T11:23:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44940ac960 0x7f44940acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4494156570 0x7f44941569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44940a8dc0 con 0x7f44940ab300 2023-10-06T11:23:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940ac5f0 con 0x7f4494156570 2023-10-06T11:23:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4494156f00 con 0x7f44940ac960 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44940ac960 0x7f44940acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4494156570 0x7f44941569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449359e700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f44940ab300 0x7f44940ab780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:46080/0 (socket says 172.21.15.7:46080) 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449359e700 1 -- 172.21.15.7:0/1821593271 learned_addr learned my addr 172.21.15.7:0/1821593271 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 -- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44940ac960 msgr2=0x7f44940acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 --2- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44940ac960 0x7f44940acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 -- 172.21.15.7:0/1821593271 >> v1:172.21.15.50:6789/0 conn(0x7f44940ab300 legacy=0x7f44940ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 -- 172.21.15.7:0/1821593271 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940ac030 con 0x7f4494156570 2023-10-06T11:23:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f449259c700 1 --2- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4494156570 0x7f44941569c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f447c0098e0 tx=0x7f447c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7488beb0a91d304c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4491d9b700 1 -- 172.21.15.7:0/1821593271 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00b640 con 0x7f4494156570 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4491d9b700 1 -- 172.21.15.7:0/1821593271 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c00b7a0 con 0x7f4494156570 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4491d9b700 1 -- 172.21.15.7:0/1821593271 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00bb40 con 0x7f4494156570 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4494156570 msgr2=0x7f44941569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 --2- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4494156570 0x7f44941569c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f447c0098e0 tx=0x7f447c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- 172.21.15.7:0/1821593271 shutdown_connections 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 --2- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4494156570 0x7f44941569c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 --2- 172.21.15.7:0/1821593271 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44940ac960 0x7f44940acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- 172.21.15.7:0/1821593271 >> 172.21.15.7:0/1821593271 conn(0x7f4494068900 msgr2=0x7f44940cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- 172.21.15.7:0/1821593271 shutdown_connections 2023-10-06T11:23:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.387+0000 7f4499138700 1 -- 172.21.15.7:0/1821593271 wait complete. 2023-10-06T11:23:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 Processor -- start 2023-10-06T11:23:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 -- start start 2023-10-06T11:23:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 0x7f449413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 0x7f44940afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449416b650 con 0x7f44940ab300 2023-10-06T11:23:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449416b930 con 0x7f44940ac960 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4499138700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449416bc10 con 0x7f4494156570 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35414/0 (socket says 172.21.15.7:35414) 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 -- 172.21.15.7:0/2476096375 learned_addr learned my addr 172.21.15.7:0/2476096375 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f449359e700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 0x7f44940afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f449259c700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 0x7f449413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 -- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 msgr2=0x7f44940afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 0x7f44940afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 -- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 msgr2=0x7f449413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 0x7f449413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 -- 172.21.15.7:0/2476096375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009420 con 0x7f44940ab300 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f449359e700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 0x7f44940afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f449259c700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 0x7f449413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f4492d9d700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f448400ec80 tx=0x7f44840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.391+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400c850 con 0x7f44940ab300 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448400c9b0 con 0x7f44940ab300 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449416bef0 con 0x7f44940ab300 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400cd50 con 0x7f44940ab300 2023-10-06T11:23:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4494140350 con 0x7f44940ab300 2023-10-06T11:23:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449406b050 con 0x7f44940ab300 2023-10-06T11:23:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4484013680 con 0x7f44940ab300 2023-10-06T11:23:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f448affd700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 0x7f4474062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f448407d3b0 con 0x7f44940ab300 2023-10-06T11:23:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.395+0000 7f449259c700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 0x7f4474062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.399+0000 7f449259c700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 0x7f4474062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f447c0093f0 tx=0x7f447c017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.403+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f448404d6c0 con 0x7f44940ab300 2023-10-06T11:23:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.567+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f449406b050 con 0x7f44940ab300 2023-10-06T11:23:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f448affd700 1 -- 172.21.15.7:0/2476096375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f449406b050 con 0x7f44940ab300 2023-10-06T11:23:36.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 msgr2=0x7f4474062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 0x7f4474062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f447c0093f0 tx=0x7f447c017000 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 msgr2=0x7f449416c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f448400ec80 tx=0x7f44840091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 shutdown_connections 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4474060670 0x7f4474062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4494156570 0x7f44940afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44940ac960 0x7f449413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 --2- 172.21.15.7:0/2476096375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44940ab300 0x7f449416c6f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 >> 172.21.15.7:0/2476096375 conn(0x7f4494068900 msgr2=0x7f44940a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 shutdown_connections 2023-10-06T11:23:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.575+0000 7f4499138700 1 -- 172.21.15.7:0/2476096375 wait complete. 2023-10-06T11:23:36.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T11:23:36.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 Processor -- start 2023-10-06T11:23:36.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 -- start start 2023-10-06T11:23:36.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 0x7f3910163b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391006e310 con 0x7f3910158f10 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391006e7a0 con 0x7f3910158b40 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f3916f01700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391006d720 con 0x7f39100ccc60 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f39156fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3910158f10 0x7f39100cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43042/0 (socket says 172.21.15.7:43042) 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.695+0000 7f39156fe700 1 -- 172.21.15.7:0/3335843740 learned_addr learned my addr 172.21.15.7:0/3335843740 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916700700 1 --2- 172.21.15.7:0/3335843740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 0x7f3910163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460896672 0 0) 0x7f391006e310 con 0x7f3910158f10 2023-10-06T11:23:36.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f8003570 con 0x7f3910158f10 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777861254 0 0) 0x7f391006e7a0 con 0x7f3910158b40 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f391006e310 con 0x7f3910158b40 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047434539 0 0) 0x7f38f8003570 con 0x7f3910158f10 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916700700 1 -- 172.21.15.7:0/3335843740 >> v1:172.21.15.50:6789/0 conn(0x7f3910158b40 legacy=0x7f39100caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916700700 1 -- 172.21.15.7:0/3335843740 >> v1:172.21.15.7:6789/0 conn(0x7f3910158f10 legacy=0x7f39100cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916700700 1 -- 172.21.15.7:0/3335843740 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391006d9e0 con 0x7f39100ccc60 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916700700 1 --2- 172.21.15.7:0/3335843740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 0x7f3910163b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e90 tx=0x7f3908009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c26684e6a7b02c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800e070 con 0x7f39100ccc60 2023-10-06T11:23:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390800b870 con 0x7f39100ccc60 2023-10-06T11:23:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3914efd700 1 -- 172.21.15.7:0/3335843740 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904001470 con 0x7f39100ccc60 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- 172.21.15.7:0/3335843740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 msgr2=0x7f3910163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 --2- 172.21.15.7:0/3335843740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 0x7f3910163b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e90 tx=0x7f3908009780 comp rx=0 tx=0).stop 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- 172.21.15.7:0/3335843740 shutdown_connections 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 --2- 172.21.15.7:0/3335843740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f39100ccc60 0x7f3910163b60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- 172.21.15.7:0/3335843740 >> 172.21.15.7:0/3335843740 conn(0x7f3910067e30 msgr2=0x7f3910068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- 172.21.15.7:0/3335843740 shutdown_connections 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- 172.21.15.7:0/3335843740 wait complete. 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 Processor -- start 2023-10-06T11:23:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.699+0000 7f3916f01700 1 -- start start 2023-10-06T11:23:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39100ccc60 0x7f39100b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3910158f10 0x7f3910145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391015d9c0 con 0x7f3910158b40 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391015cdb0 con 0x7f39100ccc60 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391015d090 con 0x7f3910158f10 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35416/0 (socket says 172.21.15.7:35416) 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 -- 172.21.15.7:0/2736329467 learned_addr learned my addr 172.21.15.7:0/2736329467 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3915eff700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39100ccc60 0x7f39100b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916700700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3910158f10 0x7f3910145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 -- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3910158f10 msgr2=0x7f3910145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3910158f10 0x7f3910145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 -- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39100ccc60 msgr2=0x7f39100b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39100ccc60 0x7f39100b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 -- 172.21.15.7:0/2736329467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009a20 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f39156fe700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f390400cb70 tx=0x7f3904001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904007150 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39040072b0 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391015d370 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904007670 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.703+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100b58e0 con 0x7f3910158b40 2023-10-06T11:23:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391015db70 con 0x7f3910158b40 2023-10-06T11:23:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3904004a40 con 0x7f3910158b40 2023-10-06T11:23:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f390dffb700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 0x7f38fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f390407c0c0 con 0x7f3910158b40 2023-10-06T11:23:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f3915eff700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 0x7f38fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:36.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.707+0000 7f3915eff700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 0x7f38fc062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3900007fd0 tx=0x7f3900007d50 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.715+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f391015db70 con 0x7f3910158b40 2023-10-06T11:23:36.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:36.931+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f391015db00 con 0x7f3910158b40 2023-10-06T11:23:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.615+0000 7f390dffb700 1 -- 172.21.15.7:0/2736329467 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v105) v1 ==== 155+0+0 (secure 0 0 0) 0x7f390404fee0 con 0x7f3910158b40 2023-10-06T11:23:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 msgr2=0x7f38fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 0x7f38fc062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3900007fd0 tx=0x7f3900007d50 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 msgr2=0x7f3910159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f390400cb70 tx=0x7f3904001ab0 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 shutdown_connections 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f38fc060670 0x7f38fc062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f3910158f10 0x7f3910145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.619+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3910158b40 0x7f3910159620 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.623+0000 7f3916f01700 1 --2- 172.21.15.7:0/2736329467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39100ccc60 0x7f39100b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.623+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 >> 172.21.15.7:0/2736329467 conn(0x7f3910067e30 msgr2=0x7f39100b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.623+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 shutdown_connections 2023-10-06T11:23:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:37.623+0000 7f3916f01700 1 -- 172.21.15.7:0/2736329467 wait complete. 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner: 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner:Ran 1 test in 52.113s 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner: 2023-10-06T11:23:37.644 INFO:tasks.cephfs_test_runner:OK 2023-10-06T11:23:37.645 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T11:23:37.656 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T11:23:37.713 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T11:23:37.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.714 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-06T11:23:37.722 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T11:23:37.722 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:23:37.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.722 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:23:37.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:23:37.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2023-10-06T11:23:37.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.964 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-06T11:23:37.972 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:23:37.972 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:23:37.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:37.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:23:38.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:38.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:23:38.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:23:38.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-10-06T11:23:38.219 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T11:23:38.229 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T11:23:38.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 Processor -- start 2023-10-06T11:23:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 -- start start 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 0x7f23cc0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0c8c20 con 0x7f23cc155840 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0c8610 con 0x7f23cc155c10 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23d1751700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15a610 con 0x7f23cc06c0b0 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23cbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f23cc155840 0x7f23cc159670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43050/0 (socket says 172.21.15.7:43050) 2023-10-06T11:23:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23cbfff700 1 -- 172.21.15.7:0/925642988 learned_addr learned my addr 172.21.15.7:0/925642988 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:38.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23cb7fe700 1 --2- 172.21.15.7:0/925642988 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 0x7f23cc0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745015351 0 0) 0x7f23cc0c8c20 con 0x7f23cc155840 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b4003570 con 0x7f23cc155840 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.339+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035639431 0 0) 0x7f23cc0c8610 con 0x7f23cc155c10 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23cc0c8c20 con 0x7f23cc155c10 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23cb7fe700 1 -- 172.21.15.7:0/925642988 >> v1:172.21.15.50:6789/0 conn(0x7f23cc155c10 legacy=0x7f23cc0c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23cb7fe700 1 -- 172.21.15.7:0/925642988 >> v1:172.21.15.7:6789/0 conn(0x7f23cc155840 legacy=0x7f23cc159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23cb7fe700 1 -- 172.21.15.7:0/925642988 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc159d80 con 0x7f23cc06c0b0 2023-10-06T11:23:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23cb7fe700 1 --2- 172.21.15.7:0/925642988 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 0x7f23cc0cc9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009910 tx=0x7f23c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a0e9d8a430171d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011630 con 0x7f23cc06c0b0 2023-10-06T11:23:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0011790 con 0x7f23cc06c0b0 2023-10-06T11:23:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23caffd700 1 -- 172.21.15.7:0/925642988 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011b30 con 0x7f23cc06c0b0 2023-10-06T11:23:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 -- 172.21.15.7:0/925642988 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 msgr2=0x7f23cc0cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 --2- 172.21.15.7:0/925642988 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 0x7f23cc0cc9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009910 tx=0x7f23c00091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 -- 172.21.15.7:0/925642988 shutdown_connections 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 --2- 172.21.15.7:0/925642988 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc06c0b0 0x7f23cc0cc9c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 -- 172.21.15.7:0/925642988 >> 172.21.15.7:0/925642988 conn(0x7f23cc06f380 msgr2=0x7f23cc06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 -- 172.21.15.7:0/925642988 shutdown_connections 2023-10-06T11:23:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.343+0000 7f23d1751700 1 -- 172.21.15.7:0/925642988 wait complete. 2023-10-06T11:23:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 Processor -- start 2023-10-06T11:23:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- start start 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc142c90 0x7f23cc144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0c3df0 con 0x7f23cc155c10 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc16a620 con 0x7f23cc155840 2023-10-06T11:23:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc169a10 con 0x7f23cc142c90 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35426/0 (socket says 172.21.15.7:35426) 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:58210/0 (socket says 172.21.15.7:58210) 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 -- 172.21.15.7:0/874641454 learned_addr learned my addr 172.21.15.7:0/874641454 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d0f50700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc142c90 0x7f23cc144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 -- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc142c90 msgr2=0x7f23cc144070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc142c90 0x7f23cc144070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 -- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 msgr2=0x7f23cc0c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 -- 172.21.15.7:0/874641454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0009420 con 0x7f23cc155c10 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cbfff700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23cb7fe700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009f00 tx=0x7f23c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c001f070 con 0x7f23cc155c10 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0002f70 con 0x7f23cc155c10 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011900 con 0x7f23cc155c10 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc169cf0 con 0x7f23cc155c10 2023-10-06T11:23:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.347+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc16a2b0 con 0x7f23cc155c10 2023-10-06T11:23:38.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.351+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc06db40 con 0x7f23cc155c10 2023-10-06T11:23:38.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.351+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f23c0023070 con 0x7f23cc155c10 2023-10-06T11:23:38.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.355+0000 7f23b3fff700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 0x7f23b8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.355+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f23c007ddc0 con 0x7f23cc155c10 2023-10-06T11:23:38.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.355+0000 7f23cbfff700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 0x7f23b8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.355+0000 7f23cbfff700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 0x7f23b8062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f23cc144960 tx=0x7f23bc009040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.363+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f23c004e0d0 con 0x7f23cc155c10 2023-10-06T11:23:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.523+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23cc15a050 con 0x7f23cc155c10 2023-10-06T11:23:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.523+0000 7f23b3fff700 1 -- 172.21.15.7:0/874641454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14686 (secure 0 0 0) 0x7f23c0015090 con 0x7f23cc155c10 2023-10-06T11:23:38.527 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:38.527 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:35.606492+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3888808928":"2023-10-07T11:23:32.597458+0000","172.21.15.7:6834/3888808928":"2023-10-07T11:23:32.597458+0000","172.21.15.50:0/1291207339":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1549220530":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/897696649":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1588488060":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3983929207":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/2318534492":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3428455953":"2023-10-07T11:23:31.992769+0000","172.21.15.50:6832/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6833/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3694782510":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3120583769":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/330210624":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/2634621881":"2023-10-07T11:23:16.399486+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.523+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 msgr2=0x7f23b8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.523+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 0x7f23b8062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f23cc144960 tx=0x7f23bc009040 comp rx=0 tx=0).stop 2023-10-06T11:23:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 msgr2=0x7f23cc143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009f00 tx=0x7f23c0017000 comp rx=0 tx=0).stop 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 shutdown_connections 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f23b8060670 0x7f23b8062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f23cc142c90 0x7f23cc144070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23cc155c10 0x7f23cc143b30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 --2- 172.21.15.7:0/874641454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23cc155840 0x7f23cc0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 >> 172.21.15.7:0/874641454 conn(0x7f23cc06f380 msgr2=0x7f23cc0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 shutdown_connections 2023-10-06T11:23:38.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.527+0000 7f23d1751700 1 -- 172.21.15.7:0/874641454 wait complete. 2023-10-06T11:23:38.541 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T11:23:38.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.647+0000 7fa142240700 1 Processor -- start 2023-10-06T11:23:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.647+0000 7fa142240700 1 -- start start 2023-10-06T11:23:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 0x7fa13c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c06d7b0 con 0x7fa13c069930 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c15a4d0 con 0x7fa13c156ab0 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c06a380 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa140a3d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa13c069930 0x7fa13c1563a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43056/0 (socket says 172.21.15.7:43056) 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa140a3d700 1 -- 172.21.15.7:0/1758303174 learned_addr learned my addr 172.21.15.7:0/1758303174 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa14123e700 1 --2- 172.21.15.7:0/1758303174 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 0x7fa13c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324933085 0 0) 0x7fa13c06d7b0 con 0x7fa13c069930 2023-10-06T11:23:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa128003570 con 0x7fa13c069930 2023-10-06T11:23:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954876585 0 0) 0x7fa13c15a4d0 con 0x7fa13c156ab0 2023-10-06T11:23:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa14123e700 1 -- 172.21.15.7:0/1758303174 >> v1:172.21.15.50:6789/0 conn(0x7fa13c156ab0 legacy=0x7fa13c06dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa14123e700 1 -- 172.21.15.7:0/1758303174 >> v1:172.21.15.7:6789/0 conn(0x7fa13c069930 legacy=0x7fa13c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa14123e700 1 -- 172.21.15.7:0/1758303174 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c06a660 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa14123e700 1 --2- 172.21.15.7:0/1758303174 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 0x7fa13c0ccc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e90 tx=0x7fa12c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9c3f7f8dacb24e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00e070 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c00b920 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa13bfff700 1 -- 172.21.15.7:0/1758303174 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124001470 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- 172.21.15.7:0/1758303174 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 msgr2=0x7fa13c0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 --2- 172.21.15.7:0/1758303174 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 0x7fa13c0ccc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e90 tx=0x7fa12c009780 comp rx=0 tx=0).stop 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- 172.21.15.7:0/1758303174 shutdown_connections 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 --2- 172.21.15.7:0/1758303174 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c0cc8b0 0x7fa13c0ccc80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- 172.21.15.7:0/1758303174 >> 172.21.15.7:0/1758303174 conn(0x7fa13c067e30 msgr2=0x7fa13c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- 172.21.15.7:0/1758303174 shutdown_connections 2023-10-06T11:23:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.651+0000 7fa142240700 1 -- 172.21.15.7:0/1758303174 wait complete. 2023-10-06T11:23:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 Processor -- start 2023-10-06T11:23:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 -- start start 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c069930 0x7fa13c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c156ab0 0x7fa13c14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0d0460 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.655+0000 7fa142240700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0b2e20 con 0x7fa13c069930 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa142240700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0cfbc0 con 0x7fa13c156ab0 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35438/0 (socket says 172.21.15.7:35438) 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 -- 172.21.15.7:0/1660407820 learned_addr learned my addr 172.21.15.7:0/1660407820 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa141a3f700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c156ab0 0x7fa13c14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa14123e700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c069930 0x7fa13c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 -- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c156ab0 msgr2=0x7fa13c14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c156ab0 0x7fa13c14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 -- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c069930 msgr2=0x7fa13c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c069930 0x7fa13c0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 -- 172.21.15.7:0/1660407820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c009a20 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa140a3d700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1240015a0 tx=0x7fa124001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124007360 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240074e0 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0cfea0 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124004830 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c0cf8d0 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c06ca60 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa124004c00 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 0x7fa11c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.659+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fa12407c410 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.663+0000 7fa14123e700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 0x7fa11c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.663+0000 7fa14123e700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 0x7fa11c062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009de0 tx=0x7fa12c017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.667+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa12404c780 con 0x7fa13c0cc8b0 2023-10-06T11:23:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.827+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa13c0b2f60 con 0x7fa11c060670 2023-10-06T11:23:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.835+0000 7fa1397fa700 1 -- 172.21.15.7:0/1660407820 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71027 (secure 0 0 0) 0x7fa13c0b2f60 con 0x7fa11c060670 2023-10-06T11:23:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 msgr2=0x7fa11c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 0x7fa11c062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009de0 tx=0x7fa12c017000 comp rx=0 tx=0).stop 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 msgr2=0x7fa13c157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1240015a0 tx=0x7fa124001bf0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 shutdown_connections 2023-10-06T11:23:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa11c060670 0x7fa11c062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa13c156ab0 0x7fa13c14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13c0cc8b0 0x7fa13c157c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 --2- 172.21.15.7:0/1660407820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c069930 0x7fa13c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 >> 172.21.15.7:0/1660407820 conn(0x7fa13c067e30 msgr2=0x7fa13c06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 shutdown_connections 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.839+0000 7fa142240700 1 -- 172.21.15.7:0/1660407820 wait complete. 2023-10-06T11:23:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:38.858 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-06T11:23:38.016528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417332,"kb_used_data":6260,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752557388,"statfs":{"total":773094113280,"available":770618765312,"internally_reserved":0,"allocated":6410240,"data_stored":4256125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992355"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.442605+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:16.442605+0000","last_peered":"2023-10-06T11:23:16.442605+0000","last_clean":"2023-10-06T11:23:16.442605+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:16.442605+0000","last_undegraded":"2023-10-06T11:23:16.442605+0000","last_fullsized":"2023-10-06T11:23:16.442605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.059212+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:32.059212+0000","last_peered":"2023-10-06T11:23:32.059212+0000","last_clean":"2023-10-06T11:23:32.059212+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:32.059212+0000","last_undegraded":"2023-10-06T11:23:32.059212+0000","last_fullsized":"2023-10-06T11:23:32.059212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.443022+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:16.443022+0000","last_peered":"2023-10-06T11:23:16.443022+0000","last_clean":"2023-10-06T11:23:16.443022+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:16.443022+0000","last_undegraded":"2023-10-06T11:23:16.443022+0000","last_fullsized":"2023-10-06T11:23:16.443022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.032811+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:32.032811+0000","last_peered":"2023-10-06T11:23:32.032811+0000","last_clean":"2023-10-06T11:23:32.032811+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:32.032811+0000","last_undegraded":"2023-10-06T11:23:32.032811+0000","last_fullsized":"2023-10-06T11:23:32.032811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'139","reported_seq":259,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.089397+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:32.089397+0000","last_peered":"2023-10-06T11:23:32.089397+0000","last_clean":"2023-10-06T11:23:32.089397+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:32.089397+0000","last_undegraded":"2023-10-06T11:23:32.089397+0000","last_fullsized":"2023-10-06T11:23:32.089397+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.409280+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:16.409280+0000","last_peered":"2023-10-06T11:23:16.409280+0000","last_clean":"2023-10-06T11:23:16.409280+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:16.409280+0000","last_undegraded":"2023-10-06T11:23:16.409280+0000","last_fullsized":"2023-10-06T11:23:16.409280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.010800+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:32.010800+0000","last_peered":"2023-10-06T11:23:32.010800+0000","last_clean":"2023-10-06T11:23:32.010800+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:32.010800+0000","last_undegraded":"2023-10-06T11:23:32.010800+0000","last_fullsized":"2023-10-06T11:23:32.010800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582070+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:16.582070+0000","last_peered":"2023-10-06T11:23:16.582070+0000","last_clean":"2023-10-06T11:23:16.582070+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:16.582070+0000","last_undegraded":"2023-10-06T11:23:16.582070+0000","last_fullsized":"2023-10-06T11:23:16.582070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582403+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:16.582403+0000","last_peered":"2023-10-06T11:23:16.582403+0000","last_clean":"2023-10-06T11:23:16.582403+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:16.582403+0000","last_undegraded":"2023-10-06T11:23:16.582403+0000","last_fullsized":"2023-10-06T11:23:16.582403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":807063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.874},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.032}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.179},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.357}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.258},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.2250000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":217350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.23}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.345}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.319},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.276}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.423},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.3740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":552208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.2}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.28},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.03},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.095},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.389}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.355}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":555126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.871},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302180,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069660,"statfs":{"total":96636764160,"available":96327331840,"internally_reserved":0,"allocated":815104,"data_stored":544895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.107}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.0269999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":227468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.127},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.079},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":1135376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.198}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.276},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.115},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.859}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.238}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.734}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":479232,"data_stored":216639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.995}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.89800000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.04},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.97999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:38.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 Processor -- start 2023-10-06T11:23:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 -- start start 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cdc15af50 0x7f1cdc0af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1cdc0af8c0 0x7f1cdc15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15b3c0 con 0x7f1cdc15ab80 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15f540 con 0x7f1cdc15af50 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce29b9700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15e950 con 0x7f1cdc0af8c0 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cdc15af50 0x7f1cdc0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce19b7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1cdc15ab80 0x7f1cdc0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43066/0 (socket says 172.21.15.7:43066) 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce19b7700 1 -- 172.21.15.7:0/131983957 learned_addr learned my addr 172.21.15.7:0/131983957 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce21b8700 1 --2- 172.21.15.7:0/131983957 >> v2:172.21.15.50:3300/0 conn(0x7f1cdc0af8c0 0x7f1cdc15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 -- 172.21.15.7:0/131983957 >> v2:172.21.15.50:3300/0 conn(0x7f1cdc0af8c0 msgr2=0x7f1cdc15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 --2- 172.21.15.7:0/131983957 >> v2:172.21.15.50:3300/0 conn(0x7f1cdc0af8c0 0x7f1cdc15f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 -- 172.21.15.7:0/131983957 >> v1:172.21.15.7:6789/0 conn(0x7f1cdc15ab80 legacy=0x7f1cdc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 -- 172.21.15.7:0/131983957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc15ec10 con 0x7f1cdc15af50 2023-10-06T11:23:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce11b6700 1 --2- 172.21.15.7:0/131983957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15af50 0x7f1cdc0af380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06bc90 tx=0x7f1cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180caf6e71709bc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce09b5700 1 -- 172.21.15.7:0/131983957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000b6b0 con 0x7f1cdc15af50 2023-10-06T11:23:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce09b5700 1 -- 172.21.15.7:0/131983957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000b810 con 0x7f1cdc15af50 2023-10-06T11:23:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.967+0000 7f1ce09b5700 1 -- 172.21.15.7:0/131983957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000bbb0 con 0x7f1cdc15af50 2023-10-06T11:23:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/131983957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15af50 msgr2=0x7f1cdc0af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/131983957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15af50 0x7f1cdc0af380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06bc90 tx=0x7f1cd00091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/131983957 shutdown_connections 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/131983957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15af50 0x7f1cdc0af380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/131983957 >> v2:172.21.15.50:3300/0 conn(0x7f1cdc0af8c0 0x7f1cdc15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/131983957 >> 172.21.15.7:0/131983957 conn(0x7f1cdc0688a0 msgr2=0x7f1cdc0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/131983957 shutdown_connections 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/131983957 wait complete. 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 Processor -- start 2023-10-06T11:23:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- start start 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cdc0af8c0 0x7f1cdc0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1cdc15af50 0x7f1cdc170e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0ad390 con 0x7f1cdc15ab80 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc14b110 con 0x7f1cdc0af8c0 2023-10-06T11:23:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc14cc40 con 0x7f1cdc15af50 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35454/0 (socket says 172.21.15.7:35454) 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 -- 172.21.15.7:0/1139650724 learned_addr learned my addr 172.21.15.7:0/1139650724 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce19b7700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cdc0af8c0 0x7f1cdc0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce21b8700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1cdc15af50 0x7f1cdc170e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 -- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1cdc15af50 msgr2=0x7f1cdc170e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1cdc15af50 0x7f1cdc170e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 -- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cdc0af8c0 msgr2=0x7f1cdc0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cdc0af8c0 0x7f1cdc0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 -- 172.21.15.7:0/1139650724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009420 con 0x7f1cdc15ab80 2023-10-06T11:23:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce11b6700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06ccd0 tx=0x7f1cd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0000d10 con 0x7f1cdc15ab80 2023-10-06T11:23:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000bb90 con 0x7f1cdc15ab80 2023-10-06T11:23:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc14a500 con 0x7f1cdc15ab80 2023-10-06T11:23:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011670 con 0x7f1cdc15ab80 2023-10-06T11:23:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.971+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc14aac0 con 0x7f1cdc15ab80 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc14b2a0 con 0x7f1cdc15ab80 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1cd0011aa0 con 0x7f1cdc15ab80 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ccdffb700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f1cd007e430 con 0x7f1cdc15ab80 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ce19b7700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.975+0000 7f1ce19b7700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 0x7f1cc0062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8007130 tx=0x7f1cd8007eb0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:38.983+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1cd004e740 con 0x7f1cdc15ab80 2023-10-06T11:23:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.139+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cdc15eef0 con 0x7f1cc0060670 2023-10-06T11:23:39.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.143+0000 7f1ccdffb700 1 -- 172.21.15.7:0/1139650724 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71027 (secure 0 0 0) 0x7f1cdc15eef0 con 0x7f1cc0060670 2023-10-06T11:23:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 msgr2=0x7f1cc0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 0x7f1cc0062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8007130 tx=0x7f1cd8007eb0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 msgr2=0x7f1cdc1708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06ccd0 tx=0x7f1cd0017000 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 shutdown_connections 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1cdc15af50 0x7f1cdc170e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc15ab80 0x7f1cdc1708c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 --2- 172.21.15.7:0/1139650724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cdc0af8c0 0x7f1cdc0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 >> 172.21.15.7:0/1139650724 conn(0x7f1cdc0688a0 msgr2=0x7f1cdc15b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 shutdown_connections 2023-10-06T11:23:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.147+0000 7f1ce29b9700 1 -- 172.21.15.7:0/1139650724 wait complete. 2023-10-06T11:23:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:39.165 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-06T11:23:38.016528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417332,"kb_used_data":6260,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752557388,"statfs":{"total":773094113280,"available":770618765312,"internally_reserved":0,"allocated":6410240,"data_stored":4256125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992355"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.442605+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:16.442605+0000","last_peered":"2023-10-06T11:23:16.442605+0000","last_clean":"2023-10-06T11:23:16.442605+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:16.442605+0000","last_undegraded":"2023-10-06T11:23:16.442605+0000","last_fullsized":"2023-10-06T11:23:16.442605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.059212+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:32.059212+0000","last_peered":"2023-10-06T11:23:32.059212+0000","last_clean":"2023-10-06T11:23:32.059212+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:32.059212+0000","last_undegraded":"2023-10-06T11:23:32.059212+0000","last_fullsized":"2023-10-06T11:23:32.059212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.443022+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:16.443022+0000","last_peered":"2023-10-06T11:23:16.443022+0000","last_clean":"2023-10-06T11:23:16.443022+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:16.443022+0000","last_undegraded":"2023-10-06T11:23:16.443022+0000","last_fullsized":"2023-10-06T11:23:16.443022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.032811+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:32.032811+0000","last_peered":"2023-10-06T11:23:32.032811+0000","last_clean":"2023-10-06T11:23:32.032811+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:32.032811+0000","last_undegraded":"2023-10-06T11:23:32.032811+0000","last_fullsized":"2023-10-06T11:23:32.032811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'139","reported_seq":259,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.089397+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:32.089397+0000","last_peered":"2023-10-06T11:23:32.089397+0000","last_clean":"2023-10-06T11:23:32.089397+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:32.089397+0000","last_undegraded":"2023-10-06T11:23:32.089397+0000","last_fullsized":"2023-10-06T11:23:32.089397+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.409280+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:16.409280+0000","last_peered":"2023-10-06T11:23:16.409280+0000","last_clean":"2023-10-06T11:23:16.409280+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:16.409280+0000","last_undegraded":"2023-10-06T11:23:16.409280+0000","last_fullsized":"2023-10-06T11:23:16.409280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.010800+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:32.010800+0000","last_peered":"2023-10-06T11:23:32.010800+0000","last_clean":"2023-10-06T11:23:32.010800+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:32.010800+0000","last_undegraded":"2023-10-06T11:23:32.010800+0000","last_fullsized":"2023-10-06T11:23:32.010800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582070+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:16.582070+0000","last_peered":"2023-10-06T11:23:16.582070+0000","last_clean":"2023-10-06T11:23:16.582070+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:16.582070+0000","last_undegraded":"2023-10-06T11:23:16.582070+0000","last_fullsized":"2023-10-06T11:23:16.582070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582403+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:16.582403+0000","last_peered":"2023-10-06T11:23:16.582403+0000","last_clean":"2023-10-06T11:23:16.582403+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:16.582403+0000","last_undegraded":"2023-10-06T11:23:16.582403+0000","last_fullsized":"2023-10-06T11:23:16.582403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":807063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.874},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.032}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.179},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.357}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.258},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.2250000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":217350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.23}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.345}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.319},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.276}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.423},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.3740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":552208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.2}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.28},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.03},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.095},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.389}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.355}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":555126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.871},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302180,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069660,"statfs":{"total":96636764160,"available":96327331840,"internally_reserved":0,"allocated":815104,"data_stored":544895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.107}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.0269999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":227468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.127},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.079},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":1135376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.198}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.276},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.115},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.859}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.238}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.734}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":479232,"data_stored":216639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.995}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.89800000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.04},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.97999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:39.167 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T11:23:39.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:39.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 Processor -- start 2023-10-06T11:23:39.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 -- start start 2023-10-06T11:23:39.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b0158b40 0x7fb1b00caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1b00ccc60 0x7fb1b0069720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b006e310 con 0x7fb1b00cb1e0 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b006e7a0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b543c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b006d720 con 0x7fb1b0158b40 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1b00ccc60 0x7fb1b0069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1af7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb1b00cb1e0 0x7fb1b00cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43078/0 (socket says 172.21.15.7:43078) 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1b00ccc60 0x7fb1b0069720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35460/0 (socket says 172.21.15.7:35460) 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1af7fe700 1 -- 172.21.15.7:0/3073097777 learned_addr learned my addr 172.21.15.7:0/3073097777 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1affff700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b0158b40 0x7fb1b00caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 -- 172.21.15.7:0/3073097777 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b0158b40 msgr2=0x7fb1b00caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b0158b40 0x7fb1b00caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 -- 172.21.15.7:0/3073097777 >> v1:172.21.15.7:6789/0 conn(0x7fb1b00cb1e0 legacy=0x7fb1b00cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1aeffd700 1 -- 172.21.15.7:0/3073097777 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104300644 0 0) 0x7fb1b006e310 con 0x7fb1b00cb1e0 2023-10-06T11:23:39.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 -- 172.21.15.7:0/3073097777 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b006d9e0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.267+0000 7fb1b4c3b700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b0069720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009e60 tx=0x7fb1a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9167cc9325b2bc2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1aeffd700 1 -- 172.21.15.7:0/3073097777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800e070 con 0x7fb1b00ccc60 2023-10-06T11:23:39.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1aeffd700 1 -- 172.21.15.7:0/3073097777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a800bb60 con 0x7fb1b00ccc60 2023-10-06T11:23:39.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1aeffd700 1 -- 172.21.15.7:0/3073097777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8012430 con 0x7fb1b00ccc60 2023-10-06T11:23:39.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- 172.21.15.7:0/3073097777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 msgr2=0x7fb1b0069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b0069720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009e60 tx=0x7fb1a8009780 comp rx=0 tx=0).stop 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- 172.21.15.7:0/3073097777 shutdown_connections 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b0069720 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- 172.21.15.7:0/3073097777 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b0158b40 0x7fb1b00caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- 172.21.15.7:0/3073097777 >> 172.21.15.7:0/3073097777 conn(0x7fb1b0067e30 msgr2=0x7fb1b0068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- 172.21.15.7:0/3073097777 shutdown_connections 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- 172.21.15.7:0/3073097777 wait complete. 2023-10-06T11:23:39.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 Processor -- start 2023-10-06T11:23:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- start start 2023-10-06T11:23:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b00cb1e0 0x7fb1b01458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1b0158b40 0x7fb1b00cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0145de0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0155a90 con 0x7fb1b0158b40 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.271+0000 7fb1b543c700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0155160 con 0x7fb1b00cb1e0 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1affff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b00cb1e0 0x7fb1b01458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35464/0 (socket says 172.21.15.7:35464) 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 -- 172.21.15.7:0/796441884 learned_addr learned my addr 172.21.15.7:0/796441884 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:39.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1b4c3b700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1b0158b40 0x7fb1b00cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 -- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b00cb1e0 msgr2=0x7fb1b01458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b00cb1e0 0x7fb1b01458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 -- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1b0158b40 msgr2=0x7fb1b00cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1b0158b40 0x7fb1b00cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 -- 172.21.15.7:0/796441884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009a20 con 0x7fb1b00ccc60 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1af7fe700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c7e0 tx=0x7fb198007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198005b30 con 0x7fb1b00ccc60 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198005c90 con 0x7fb1b00ccc60 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198007300 con 0x7fb1b00ccc60 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b0155440 con 0x7fb1b00ccc60 2023-10-06T11:23:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b0155f20 con 0x7fb1b00ccc60 2023-10-06T11:23:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb1980076b0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.275+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b006ca60 con 0x7fb1b00ccc60 2023-10-06T11:23:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.279+0000 7fb1a7fff700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 0x7fb190062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.279+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fb19807bfa0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.279+0000 7fb1affff700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 0x7fb190062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.279+0000 7fb1affff700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 0x7fb190062b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009d10 tx=0x7fb1a00091a0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.283+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb19804c2b0 con 0x7fb1b00ccc60 2023-10-06T11:23:39.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.443+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1b0155bd0 con 0x7fb190060670 2023-10-06T11:23:39.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.451+0000 7fb1a7fff700 1 -- 172.21.15.7:0/796441884 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71027 (secure 0 0 0) 0x7fb1b0155bd0 con 0x7fb190060670 2023-10-06T11:23:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 msgr2=0x7fb190062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 0x7fb190062b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009d10 tx=0x7fb1a00091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 msgr2=0x7fb1b00cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c7e0 tx=0x7fb198007df0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 shutdown_connections 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb190060670 0x7fb190062b20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1b0158b40 0x7fb1b00cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1b00ccc60 0x7fb1b00cea50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 --2- 172.21.15.7:0/796441884 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1b00cb1e0 0x7fb1b01458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 >> 172.21.15.7:0/796441884 conn(0x7fb1b0067e30 msgr2=0x7fb1b015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 shutdown_connections 2023-10-06T11:23:39.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.455+0000 7fb1b543c700 1 -- 172.21.15.7:0/796441884 wait complete. 2023-10-06T11:23:39.462 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:39.476 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-06T11:23:38.016528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417332,"kb_used_data":6260,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752557388,"statfs":{"total":773094113280,"available":770618765312,"internally_reserved":0,"allocated":6410240,"data_stored":4256125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992355"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.442605+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:16.442605+0000","last_peered":"2023-10-06T11:23:16.442605+0000","last_clean":"2023-10-06T11:23:16.442605+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:16.442605+0000","last_undegraded":"2023-10-06T11:23:16.442605+0000","last_fullsized":"2023-10-06T11:23:16.442605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.059212+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:32.059212+0000","last_peered":"2023-10-06T11:23:32.059212+0000","last_clean":"2023-10-06T11:23:32.059212+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:32.059212+0000","last_undegraded":"2023-10-06T11:23:32.059212+0000","last_fullsized":"2023-10-06T11:23:32.059212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.443022+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:16.443022+0000","last_peered":"2023-10-06T11:23:16.443022+0000","last_clean":"2023-10-06T11:23:16.443022+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:16.443022+0000","last_undegraded":"2023-10-06T11:23:16.443022+0000","last_fullsized":"2023-10-06T11:23:16.443022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.032811+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:32.032811+0000","last_peered":"2023-10-06T11:23:32.032811+0000","last_clean":"2023-10-06T11:23:32.032811+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:32.032811+0000","last_undegraded":"2023-10-06T11:23:32.032811+0000","last_fullsized":"2023-10-06T11:23:32.032811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'139","reported_seq":259,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.089397+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:32.089397+0000","last_peered":"2023-10-06T11:23:32.089397+0000","last_clean":"2023-10-06T11:23:32.089397+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:32.089397+0000","last_undegraded":"2023-10-06T11:23:32.089397+0000","last_fullsized":"2023-10-06T11:23:32.089397+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.409280+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:16.409280+0000","last_peered":"2023-10-06T11:23:16.409280+0000","last_clean":"2023-10-06T11:23:16.409280+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:16.409280+0000","last_undegraded":"2023-10-06T11:23:16.409280+0000","last_fullsized":"2023-10-06T11:23:16.409280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T11:23:32.010800+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:32.010800+0000","last_peered":"2023-10-06T11:23:32.010800+0000","last_clean":"2023-10-06T11:23:32.010800+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:32.010800+0000","last_undegraded":"2023-10-06T11:23:32.010800+0000","last_fullsized":"2023-10-06T11:23:32.010800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582070+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:16.582070+0000","last_peered":"2023-10-06T11:23:16.582070+0000","last_clean":"2023-10-06T11:23:16.582070+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:16.582070+0000","last_undegraded":"2023-10-06T11:23:16.582070+0000","last_fullsized":"2023-10-06T11:23:16.582070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-06T11:23:16.582403+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:16.582403+0000","last_peered":"2023-10-06T11:23:16.582403+0000","last_clean":"2023-10-06T11:23:16.582403+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:16.582403+0000","last_undegraded":"2023-10-06T11:23:16.582403+0000","last_fullsized":"2023-10-06T11:23:16.582403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":807063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.874},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.032}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.179},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.357}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.258},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.2250000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301860,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069980,"statfs":{"total":96636764160,"available":96327659520,"internally_reserved":0,"allocated":487424,"data_stored":217350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.23}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.345}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.319},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.276}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.423},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.3740000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":552208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.2}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.28},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.03},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.095},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.389}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.355}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":823296,"data_stored":555126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.871},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302180,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069660,"statfs":{"total":96636764160,"available":96327331840,"internally_reserved":0,"allocated":815104,"data_stored":544895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.107}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.0269999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":227468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.127},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.079},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.194}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1409024,"data_stored":1135376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.198}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.276},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.115},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.859}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.238}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.734}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":479232,"data_stored":216639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.995}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.89800000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.04},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.97999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:39.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:23:39.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.579+0000 7f1bc62b7700 1 Processor -- start 2023-10-06T11:23:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.579+0000 7f1bc62b7700 1 -- start start 2023-10-06T11:23:39.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 0x7f1bc00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1bc00b11c0 0x7f1bc015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00b07f0 con 0x7f1bc0158df0 2023-10-06T11:23:39.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00b1640 con 0x7f1bc00b11c0 2023-10-06T11:23:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00afbe0 con 0x7f1bc00af700 2023-10-06T11:23:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 0x7f1bc00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbffff700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1bc0158df0 0x7f1bc00b3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:46128/0 (socket says 172.21.15.7:46128) 2023-10-06T11:23:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbffff700 1 -- 172.21.15.7:0/1781421945 learned_addr learned my addr 172.21.15.7:0/1781421945 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc4854700 1 --2- 172.21.15.7:0/1781421945 >> v2:172.21.15.50:3300/0 conn(0x7f1bc00b11c0 0x7f1bc015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/1781421945 >> v2:172.21.15.50:3300/0 conn(0x7f1bc00b11c0 msgr2=0x7f1bc015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 --2- 172.21.15.7:0/1781421945 >> v2:172.21.15.50:3300/0 conn(0x7f1bc00b11c0 0x7f1bc015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/1781421945 >> v1:172.21.15.50:6789/0 conn(0x7f1bc0158df0 legacy=0x7f1bc00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbeffd700 1 -- 172.21.15.7:0/1781421945 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951109376 0 0) 0x7f1bc00b07f0 con 0x7f1bc0158df0 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/1781421945 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00afec0 con 0x7f1bc00af700 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbf7fe700 1 --2- 172.21.15.7:0/1781421945 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 0x7f1bc00b0c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00098e0 tx=0x7f1bb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5156d20070904fad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbeffd700 1 -- 172.21.15.7:0/1781421945 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000b640 con 0x7f1bc00af700 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbeffd700 1 -- 172.21.15.7:0/1781421945 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb000b7a0 con 0x7f1bc00af700 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bbeffd700 1 -- 172.21.15.7:0/1781421945 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000bb40 con 0x7f1bc00af700 2023-10-06T11:23:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 -- 172.21.15.7:0/1781421945 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 msgr2=0x7f1bc00b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.583+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/1781421945 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 0x7f1bc00b0c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00098e0 tx=0x7f1bb00091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- 172.21.15.7:0/1781421945 shutdown_connections 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/1781421945 >> v2:172.21.15.50:3300/0 conn(0x7f1bc00b11c0 0x7f1bc015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/1781421945 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00af700 0x7f1bc00b0c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- 172.21.15.7:0/1781421945 >> 172.21.15.7:0/1781421945 conn(0x7f1bc00688c0 msgr2=0x7f1bc00cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- 172.21.15.7:0/1781421945 shutdown_connections 2023-10-06T11:23:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- 172.21.15.7:0/1781421945 wait complete. 2023-10-06T11:23:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 Processor -- start 2023-10-06T11:23:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- start start 2023-10-06T11:23:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc00af700 0x7f1bc0153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bc0158df0 0x7f1bc015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015c560 con 0x7f1bc00af700 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00aec10 con 0x7f1bc0158df0 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc62b7700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00ae000 con 0x7f1bc00b11c0 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc00af700 0x7f1bc0153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:52114/0 (socket says 172.21.15.7:52114) 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/3018007418 learned_addr learned my addr 172.21.15.7:0/3018007418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bc4854700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bc0158df0 0x7f1bc015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bc0158df0 msgr2=0x7f1bc015c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bc0158df0 0x7f1bc015c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/3018007418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc00af700 msgr2=0x7f1bc0153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc00af700 0x7f1bc0153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.587+0000 7f1bbf7fe700 1 -- 172.21.15.7:0/3018007418 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0009420 con 0x7f1bc00b11c0 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1bbf7fe700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0013000 tx=0x7f1bb0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0000bc0 con 0x7f1bc00b11c0 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb000b910 con 0x7f1bc00b11c0 2023-10-06T11:23:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc00ae2e0 con 0x7f1bc00b11c0 2023-10-06T11:23:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0000d20 con 0x7f1bc00b11c0 2023-10-06T11:23:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc00ae8a0 con 0x7f1bc00b11c0 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc004be00 con 0x7f1bc00b11c0 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1bb0023070 con 0x7f1bc00b11c0 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 0x7f1bac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.591+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f1bb007e030 con 0x7f1bc00b11c0 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.595+0000 7f1bbffff700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 0x7f1bac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.595+0000 7f1bbffff700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 0x7f1bac062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4007160 tx=0x7f1bb4007f50 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.607+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1bb004e380 con 0x7f1bc00b11c0 2023-10-06T11:23:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.767+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc00b01a0 con 0x7f1bc00b11c0 2023-10-06T11:23:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1ba7fff700 1 -- 172.21.15.7:0/3018007418 <== mon.2 v2:172.21.15.50:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14686 (secure 0 0 0) 0x7f1bb0051960 con 0x7f1bc00b11c0 2023-10-06T11:23:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:39.774 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"88789558-6143-4e37-9ef9-c1260e585533","created":"2023-10-06T11:21:39.479954+0000","modified":"2023-10-06T11:23:35.606492+0000","last_up_change":"2023-10-06T11:21:50.141043+0000","last_in_change":"2023-10-06T11:21:42.194948+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:21:50.897583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:21:51.875482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7b851f7e-bc0d-49e3-966e-45a3192e5cdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":33385},{"type":"v1","addr":"172.21.15.7:6809","nonce":33385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":33385},{"type":"v1","addr":"172.21.15.7:6811","nonce":33385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":33385},{"type":"v1","addr":"172.21.15.7:6815","nonce":33385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":33385},{"type":"v1","addr":"172.21.15.7:6813","nonce":33385}]},"public_addr":"172.21.15.7:6809/33385","cluster_addr":"172.21.15.7:6811/33385","heartbeat_back_addr":"172.21.15.7:6815/33385","heartbeat_front_addr":"172.21.15.7:6813/33385","state":["exists","up"]},{"osd":1,"uuid":"bbf26255-a037-41f7-a74f-fa7274055679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":33388},{"type":"v1","addr":"172.21.15.7:6817","nonce":33388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":33388},{"type":"v1","addr":"172.21.15.7:6819","nonce":33388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":33388},{"type":"v1","addr":"172.21.15.7:6823","nonce":33388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":33388},{"type":"v1","addr":"172.21.15.7:6821","nonce":33388}]},"public_addr":"172.21.15.7:6817/33388","cluster_addr":"172.21.15.7:6819/33388","heartbeat_back_addr":"172.21.15.7:6823/33388","heartbeat_front_addr":"172.21.15.7:6821/33388","state":["exists","up"]},{"osd":2,"uuid":"ab91019f-a131-455c-9f84-862984c7ee84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":33406},{"type":"v1","addr":"172.21.15.7:6825","nonce":33406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":33406},{"type":"v1","addr":"172.21.15.7:6827","nonce":33406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":33406},{"type":"v1","addr":"172.21.15.7:6831","nonce":33406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":33406},{"type":"v1","addr":"172.21.15.7:6829","nonce":33406}]},"public_addr":"172.21.15.7:6825/33406","cluster_addr":"172.21.15.7:6827/33406","heartbeat_back_addr":"172.21.15.7:6831/33406","heartbeat_front_addr":"172.21.15.7:6829/33406","state":["exists","up"]},{"osd":3,"uuid":"399b2d85-efb6-4838-93ed-3010a09e786e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":33384},{"type":"v1","addr":"172.21.15.7:6801","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":33384},{"type":"v1","addr":"172.21.15.7:6803","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":33384},{"type":"v1","addr":"172.21.15.7:6807","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":33384},{"type":"v1","addr":"172.21.15.7:6805","nonce":33384}]},"public_addr":"172.21.15.7:6801/33384","cluster_addr":"172.21.15.7:6803/33384","heartbeat_back_addr":"172.21.15.7:6807/33384","heartbeat_front_addr":"172.21.15.7:6805/33384","state":["exists","up"]},{"osd":4,"uuid":"ea94ba68-eeed-4571-b2fc-3d175337222f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":33665},{"type":"v1","addr":"172.21.15.50:6801","nonce":33665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":33665},{"type":"v1","addr":"172.21.15.50:6803","nonce":33665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":33665},{"type":"v1","addr":"172.21.15.50:6807","nonce":33665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":33665},{"type":"v1","addr":"172.21.15.50:6805","nonce":33665}]},"public_addr":"172.21.15.50:6801/33665","cluster_addr":"172.21.15.50:6803/33665","heartbeat_back_addr":"172.21.15.50:6807/33665","heartbeat_front_addr":"172.21.15.50:6805/33665","state":["exists","up"]},{"osd":5,"uuid":"ddad3fd1-81da-47a1-8d35-5c213d4051da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":33666},{"type":"v1","addr":"172.21.15.50:6818","nonce":33666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":33666},{"type":"v1","addr":"172.21.15.50:6821","nonce":33666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6827","nonce":33666},{"type":"v1","addr":"172.21.15.50:6829","nonce":33666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6823","nonce":33666},{"type":"v1","addr":"172.21.15.50:6825","nonce":33666}]},"public_addr":"172.21.15.50:6818/33666","cluster_addr":"172.21.15.50:6821/33666","heartbeat_back_addr":"172.21.15.50:6829/33666","heartbeat_front_addr":"172.21.15.50:6825/33666","state":["exists","up"]},{"osd":6,"uuid":"9f702162-6deb-48cb-8a8b-d1518c0b3420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":33664},{"type":"v1","addr":"172.21.15.50:6809","nonce":33664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":33664},{"type":"v1","addr":"172.21.15.50:6811","nonce":33664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":33664},{"type":"v1","addr":"172.21.15.50:6815","nonce":33664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":33664},{"type":"v1","addr":"172.21.15.50:6813","nonce":33664}]},"public_addr":"172.21.15.50:6809/33664","cluster_addr":"172.21.15.50:6811/33664","heartbeat_back_addr":"172.21.15.50:6815/33664","heartbeat_front_addr":"172.21.15.50:6813/33664","state":["exists","up"]},{"osd":7,"uuid":"9b5682fa-9148-4801-9c00-e59ff2f62347","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":33667},{"type":"v1","addr":"172.21.15.50:6820","nonce":33667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":33667},{"type":"v1","addr":"172.21.15.50:6824","nonce":33667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":33667},{"type":"v1","addr":"172.21.15.50:6831","nonce":33667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":33667},{"type":"v1","addr":"172.21.15.50:6828","nonce":33667}]},"public_addr":"172.21.15.50:6820/33667","cluster_addr":"172.21.15.50:6824/33667","heartbeat_back_addr":"172.21.15.50:6831/33667","heartbeat_front_addr":"172.21.15.50:6828/33667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.336886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.267627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.473090+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.150589+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429028+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.429891+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.455708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:21:47.222203+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3888808928":"2023-10-07T11:23:32.597458+0000","172.21.15.7:6834/3888808928":"2023-10-07T11:23:32.597458+0000","172.21.15.50:0/1291207339":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1549220530":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/897696649":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/1588488060":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3983929207":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/2318534492":"2023-10-07T11:23:31.992769+0000","172.21.15.50:0/3428455953":"2023-10-07T11:23:31.992769+0000","172.21.15.50:6832/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:6833/33317":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3694782510":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/3120583769":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/330210624":"2023-10-07T11:23:16.399486+0000","172.21.15.50:0/2634621881":"2023-10-07T11:23:16.399486+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 msgr2=0x7f1bac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 0x7f1bac062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4007160 tx=0x7f1bb4007f50 comp rx=0 tx=0).stop 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 msgr2=0x7f1bc00ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0013000 tx=0x7f1bb0017000 comp rx=0 tx=0).stop 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 shutdown_connections 2023-10-06T11:23:39.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f1bac060670 0x7f1bac062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bc0158df0 0x7f1bc015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f1bc00b11c0 0x7f1bc00ad960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 --2- 172.21.15.7:0/3018007418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc00af700 0x7f1bc0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 >> 172.21.15.7:0/3018007418 conn(0x7f1bc00688c0 msgr2=0x7f1bc0156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 shutdown_connections 2023-10-06T11:23:39.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:39.771+0000 7f1bc62b7700 1 -- 172.21.15.7:0/3018007418 wait complete. 2023-10-06T11:23:40.790 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T11:23:40.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:40.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.911+0000 7f4cebfff700 1 Processor -- start 2023-10-06T11:23:40.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cebfff700 1 -- start start 2023-10-06T11:23:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cebfff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4cec0cd0d0 0x7f4cec0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cebfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec0c8670 con 0x7f4cec158cb0 2023-10-06T11:23:40.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cebfff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec0c7ad0 con 0x7f4cec159080 2023-10-06T11:23:40.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cebfff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cec1594d0 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cec158cb0 0x7f4cec0c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43088/0 (socket says 172.21.15.7:43088) 2023-10-06T11:23:40.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceaffd700 1 -- 172.21.15.7:0/3396348460 learned_addr learned my addr 172.21.15.7:0/3396348460 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:40.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceb7fe700 1 --2- 172.21.15.7:0/3396348460 >> v2:172.21.15.50:3300/0 conn(0x7f4cec0cd0d0 0x7f4cec0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cea7fc700 1 -- 172.21.15.7:0/3396348460 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439281851 0 0) 0x7f4cdc001110 con 0x7f4cec158cb0 2023-10-06T11:23:40.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4cea7fc700 1 -- 172.21.15.7:0/3396348460 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd8003570 con 0x7f4cec158cb0 2023-10-06T11:23:40.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceb7fe700 1 -- 172.21.15.7:0/3396348460 >> v1:172.21.15.50:6789/0 conn(0x7f4cec159080 legacy=0x7f4cec0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceb7fe700 1 -- 172.21.15.7:0/3396348460 >> v1:172.21.15.7:6789/0 conn(0x7f4cec158cb0 legacy=0x7f4cec0c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceb7fe700 1 -- 172.21.15.7:0/3396348460 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0c7210 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.915+0000 7f4ceb7fe700 1 --2- 172.21.15.7:0/3396348460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec0c9630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009e60 tx=0x7f4ce4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9499417960f9021 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cea7fc700 1 -- 172.21.15.7:0/3396348460 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400e070 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cea7fc700 1 -- 172.21.15.7:0/3396348460 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce400bb60 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cea7fc700 1 -- 172.21.15.7:0/3396348460 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4012430 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 -- 172.21.15.7:0/3396348460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec0cd0d0 msgr2=0x7f4cec0c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 --2- 172.21.15.7:0/3396348460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec0c9630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009e60 tx=0x7f4ce4009780 comp rx=0 tx=0).stop 2023-10-06T11:23:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 -- 172.21.15.7:0/3396348460 shutdown_connections 2023-10-06T11:23:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 --2- 172.21.15.7:0/3396348460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec0c9630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 -- 172.21.15.7:0/3396348460 >> 172.21.15.7:0/3396348460 conn(0x7f4cec06ea80 msgr2=0x7f4cec06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 -- 172.21.15.7:0/3396348460 shutdown_connections 2023-10-06T11:23:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.919+0000 7f4cebfff700 1 -- 172.21.15.7:0/3396348460 wait complete. 2023-10-06T11:23:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 Processor -- start 2023-10-06T11:23:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- start start 2023-10-06T11:23:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec16d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4cec158cb0 0x7f4cec148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec159080 0x7f4cec0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec16d900 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0c6330 con 0x7f4cec159080 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0c5720 con 0x7f4cec158cb0 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec16d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec16d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35482/0 (socket says 172.21.15.7:35482) 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 -- 172.21.15.7:0/248394429 learned_addr learned my addr 172.21.15.7:0/248394429 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceb7fe700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec159080 0x7f4cec0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ce3fff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4cec158cb0 0x7f4cec148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 -- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4cec158cb0 msgr2=0x7f4cec148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4cec158cb0 0x7f4cec148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 -- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec159080 msgr2=0x7f4cec0c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cec159080 0x7f4cec0c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4009a20 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ceaffd700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec16d3c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c7c0 tx=0x7f4cdc007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005c00 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc005d60 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec0c5a00 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007320 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.923+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec0c5fc0 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cec06d720 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4cdc01b3b0 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4ccc060670 0x7f4ccc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f4cdc07bee0 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f4ccc067500 0x7f4ccc069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ccc06a000 con 0x7f4ccc067500 2023-10-06T11:23:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cdc07e4c0 con 0x7f4cec0cd0d0 2023-10-06T11:23:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ce3fff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4ccc060670 0x7f4ccc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ceb7fe700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f4ccc067500 0x7f4ccc069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.927+0000 7f4ceb7fe700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f4ccc067500 0x7f4ccc069930 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.931+0000 7f4ce3fff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4ccc060670 0x7f4ccc062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4cec16c970 tx=0x7f4cd4007ea0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.931+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== osd.0 v2:172.21.15.7:6808/33385 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4ccc06a000 con 0x7f4ccc067500 2023-10-06T11:23:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.947+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 --> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4cec0c8670 con 0x7f4ccc067500 2023-10-06T11:23:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.947+0000 7f4ce2ffd700 1 -- 172.21.15.7:0/248394429 <== osd.0 v2:172.21.15.7:6808/33385 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4cec0c8670 con 0x7f4ccc067500 2023-10-06T11:23:40.953 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:40.953 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f4ccc067500 msgr2=0x7f4ccc069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:6808/33385,v1:172.21.15.7:6809/33385] conn(0x7f4ccc067500 0x7f4ccc069930 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4ccc060670 msgr2=0x7f4ccc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4ccc060670 0x7f4ccc062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4cec16c970 tx=0x7f4cd4007ea0 comp rx=0 tx=0).stop 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 msgr2=0x7f4cec16d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 --2- 172.21.15.7:0/248394429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cec0cd0d0 0x7f4cec16d3c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c7c0 tx=0x7f4cdc007c70 comp rx=0 tx=0).stop 2023-10-06T11:23:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4ceb7fe700 1 -- 172.21.15.7:0/248394429 reap_dead start 2023-10-06T11:23:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 shutdown_connections 2023-10-06T11:23:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 >> 172.21.15.7:0/248394429 conn(0x7f4cec06ea80 msgr2=0x7f4cec06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 shutdown_connections 2023-10-06T11:23:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:40.951+0000 7f4cebfff700 1 -- 172.21.15.7:0/248394429 wait complete. 2023-10-06T11:23:40.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T11:23:41.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 Processor -- start 2023-10-06T11:23:41.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 -- start start 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f44cc0ce550 0x7f44cc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc069f50 con 0x7f44cc06afd0 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc0cd440 con 0x7f44cc0ce550 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d2e5a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc06a230 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44cbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f44cc06afd0 0x7f44cc0cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43098/0 (socket says 172.21.15.7:43098) 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44cbfff700 1 -- 172.21.15.7:0/2281535779 learned_addr learned my addr 172.21.15.7:0/2281535779 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d13f7700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2281535779 >> v2:172.21.15.50:3300/0 conn(0x7f44cc0ce550 0x7f44cc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44cb7fe700 1 -- 172.21.15.7:0/2281535779 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413998988 0 0) 0x7f44cc069f50 con 0x7f44cc06afd0 2023-10-06T11:23:41.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.079+0000 7f44cb7fe700 1 -- 172.21.15.7:0/2281535779 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b8003570 con 0x7f44cc06afd0 2023-10-06T11:23:41.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d0bf6700 1 -- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 msgr2=0x7f44cc0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:41.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d0bf6700 1 -- 172.21.15.7:0/2281535779 >> v1:172.21.15.7:6789/0 conn(0x7f44cc06afd0 legacy=0x7f44cc0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d0bf6700 1 -- 172.21.15.7:0/2281535779 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc06a510 con 0x7f44cc0ce550 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d13f7700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc0ce550 0x7f44cc0ce920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098e0 tx=0x7f44b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c91ce58bebf4ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44cb7fe700 1 -- 172.21.15.7:0/2281535779 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4011630 con 0x7f44cc0ce550 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44cb7fe700 1 -- 172.21.15.7:0/2281535779 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4011790 con 0x7f44cc0ce550 2023-10-06T11:23:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44cb7fe700 1 -- 172.21.15.7:0/2281535779 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4011b30 con 0x7f44cc0ce550 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc0ce550 msgr2=0x7f44cc0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc0ce550 0x7f44cc0ce920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098e0 tx=0x7f44b40091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2281535779 shutdown_connections 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2281535779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc0ce550 0x7f44cc0ce920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2281535779 >> 172.21.15.7:0/2281535779 conn(0x7f44cc067e30 msgr2=0x7f44cc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2281535779 shutdown_connections 2023-10-06T11:23:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.083+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2281535779 wait complete. 2023-10-06T11:23:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 Processor -- start 2023-10-06T11:23:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 -- start start 2023-10-06T11:23:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc06afd0 0x7f44cc146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc146f90 con 0x7f44cc0ce550 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0b2c60 con 0x7f44cc06afd0 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d2e5a700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0b2eb0 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d13f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d13f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35494/0 (socket says 172.21.15.7:35494) 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d13f7700 1 -- 172.21.15.7:0/2317419583 learned_addr learned my addr 172.21.15.7:0/2317419583 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc06afd0 0x7f44cc146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 -- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc06afd0 msgr2=0x7f44cc146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc06afd0 0x7f44cc146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 -- 172.21.15.7:0/2317419583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 msgr2=0x7f44cc0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 -- 172.21.15.7:0/2317419583 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009a20 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44d13f7700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44cbfff700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0c89e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c790 tx=0x7f44bc007bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0072e0 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.087+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc007440 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b4009420 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0057f0 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc0b3470 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc070f20 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f44bc005c40 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44c8ff9700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 0x7f44ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f44bc07bf20 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.091+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 0x7f44ac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.095+0000 7f44d0bf6700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 0x7f44ac062b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f44b40091e0 tx=0x7f44b4017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.103+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mon.2 v2:172.21.15.50:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f44bc04c210 con 0x7f44cc0ccfc0 2023-10-06T11:23:41.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.267+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f44cc0b5c90 con 0x7f44ac060670 2023-10-06T11:23:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.267+0000 7f44c8ff9700 1 -- 172.21.15.7:0/2317419583 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44cc0b5c90 con 0x7f44ac060670 2023-10-06T11:23:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 msgr2=0x7f44ac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 0x7f44ac062b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f44b40091e0 tx=0x7f44b4017000 comp rx=0 tx=0).stop 2023-10-06T11:23:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 msgr2=0x7f44cc0c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0c89e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c790 tx=0x7f44bc007bc0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 shutdown_connections 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f44ac060670 0x7f44ac062b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc0ce550 0x7f44cc0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f44cc0ccfc0 0x7f44cc0c89e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 --2- 172.21.15.7:0/2317419583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44cc06afd0 0x7f44cc146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 >> 172.21.15.7:0/2317419583 conn(0x7f44cc067e30 msgr2=0x7f44cc150090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 shutdown_connections 2023-10-06T11:23:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.271+0000 7f44d2e5a700 1 -- 172.21.15.7:0/2317419583 wait complete. 2023-10-06T11:23:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T11:23:41.285 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T11:23:41.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:41.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.391+0000 7fb1e9eb4700 1 Processor -- start 2023-10-06T11:23:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.391+0000 7fb1e9eb4700 1 -- start start 2023-10-06T11:23:41.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e9eb4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e40ccd70 0x7fb1e40ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e9eb4700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb1e406d940 0x7fb1e40b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e9eb4700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40cac70 con 0x7fb1e4158ea0 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e9eb4700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e4068d40 con 0x7fb1e406d940 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e9eb4700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40caea0 con 0x7fb1e40ccd70 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb1e406d940 0x7fb1e40b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e40ccd70 0x7fb1e40ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb1e406d940 0x7fb1e40b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:58298/0 (socket says 172.21.15.7:58298) 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 -- 172.21.15.7:0/1825500791 learned_addr learned my addr 172.21.15.7:0/1825500791 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e8eb2700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb1e4158ea0 0x7fb1e40cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:46150/0 (socket says 172.21.15.7:46150) 2023-10-06T11:23:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 -- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e40ccd70 msgr2=0x7fb1e40ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 --2- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e40ccd70 0x7fb1e40ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 -- 172.21.15.7:0/1825500791 >> v1:172.21.15.50:6789/0 conn(0x7fb1e4158ea0 legacy=0x7fb1e40cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 -- 172.21.15.7:0/1825500791 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40cb180 con 0x7fb1e406d940 2023-10-06T11:23:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e96b3700 1 --2- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b0490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009f20 tx=0x7fb1dc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d590f7aeede86f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e37fe700 1 -- 172.21.15.7:0/1825500791 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00e070 con 0x7fb1e406d940 2023-10-06T11:23:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e37fe700 1 -- 172.21.15.7:0/1825500791 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00ba60 con 0x7fb1e406d940 2023-10-06T11:23:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.395+0000 7fb1e37fe700 1 -- 172.21.15.7:0/1825500791 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0092e0 con 0x7fb1e406d940 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 msgr2=0x7fb1e40b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b0490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009f20 tx=0x7fb1dc009840 comp rx=0 tx=0).stop 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/1825500791 shutdown_connections 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b0490 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/1825500791 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e40ccd70 0x7fb1e40ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/1825500791 >> 172.21.15.7:0/1825500791 conn(0x7fb1e4067e30 msgr2=0x7fb1e4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/1825500791 shutdown_connections 2023-10-06T11:23:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/1825500791 wait complete. 2023-10-06T11:23:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 Processor -- start 2023-10-06T11:23:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.399+0000 7fb1e9eb4700 1 -- start start 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 0x7fb1e414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e4158ea0 0x7fb1e415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b5540 con 0x7fb1e40ccd70 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b3fa0 con 0x7fb1e406d940 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b4240 con 0x7fb1e4158ea0 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 0x7fb1e414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e8eb2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 0x7fb1e414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35506/0 (socket says 172.21.15.7:35506) 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 -- 172.21.15.7:0/2279781010 learned_addr learned my addr 172.21.15.7:0/2279781010 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e96b3700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e4158ea0 0x7fb1e415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 -- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e4158ea0 msgr2=0x7fb1e415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb1e4158ea0 0x7fb1e415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 -- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 msgr2=0x7fb1e40b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc009ae0 con 0x7fb1e40ccd70 2023-10-06T11:23:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e8eb2700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e406d940 0x7fb1e40b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e3fff700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 0x7fb1e414e2b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400ec80 tx=0x7fb1d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400c850 con 0x7fb1e40ccd70 2023-10-06T11:23:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400c9b0 con 0x7fb1e40ccd70 2023-10-06T11:23:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40b4520 con 0x7fb1e40ccd70 2023-10-06T11:23:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400cd50 con 0x7fb1e40ccd70 2023-10-06T11:23:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.403+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40b4b60 con 0x7fb1e40ccd70 2023-10-06T11:23:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1e406c5c0 con 0x7fb1e40ccd70 2023-10-06T11:23:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb1d4013680 con 0x7fb1e40ccd70 2023-10-06T11:23:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb1c4060670 0x7fb1c4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fb1d407cdb0 con 0x7fb1e40ccd70 2023-10-06T11:23:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7fb1c4067500 0x7fb1c4069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1c406a000 con 0x7fb1c4067500 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d407f310 con 0x7fb1e40ccd70 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e8eb2700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb1c4060670 0x7fb1c4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e96b3700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7fb1c4067500 0x7fb1c4069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e96b3700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7fb1c4067500 0x7fb1c4069930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.407+0000 7fb1e8eb2700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb1c4060670 0x7fb1c4062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1e415e940 tx=0x7fb1cc007eb0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.411+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== osd.1 v2:172.21.15.7:6816/33388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb1c406a000 con 0x7fb1c4067500 2023-10-06T11:23:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 --> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb1e40cb460 con 0x7fb1c4067500 2023-10-06T11:23:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e0ff9700 1 -- 172.21.15.7:0/2279781010 <== osd.1 v2:172.21.15.7:6816/33388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb1e40cb460 con 0x7fb1c4067500 2023-10-06T11:23:41.431 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:41.431 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:41.431 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7fb1c4067500 msgr2=0x7fb1c4069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:6816/33388,v1:172.21.15.7:6817/33388] conn(0x7fb1c4067500 0x7fb1c4069930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb1c4060670 msgr2=0x7fb1c4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb1c4060670 0x7fb1c4062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1e415e940 tx=0x7fb1cc007eb0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 msgr2=0x7fb1e414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 --2- 172.21.15.7:0/2279781010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e40ccd70 0x7fb1e414e2b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400ec80 tx=0x7fb1d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e96b3700 1 -- 172.21.15.7:0/2279781010 reap_dead start 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 shutdown_connections 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 >> 172.21.15.7:0/2279781010 conn(0x7fb1e4067e30 msgr2=0x7fb1e414ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 shutdown_connections 2023-10-06T11:23:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.427+0000 7fb1e9eb4700 1 -- 172.21.15.7:0/2279781010 wait complete. 2023-10-06T11:23:41.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T11:23:41.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 Processor -- start 2023-10-06T11:23:41.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 -- start start 2023-10-06T11:23:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9dc0ade60 0x7fa9dc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9dc0c88e0 0x7fa9dc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0c8d40 con 0x7fa9dc0c88e0 2023-10-06T11:23:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc15da30 con 0x7fa9dc0ade60 2023-10-06T11:23:41.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e3449700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc15a490 con 0x7fa9dc0ae230 2023-10-06T11:23:41.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9dc0ade60 0x7fa9dc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e19e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9dc0c88e0 0x7fa9dc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e09e4700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9dc0ade60 0x7fa9dc159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:58320/0 (socket says 172.21.15.7:58320) 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 -- 172.21.15.7:0/266913671 learned_addr learned my addr 172.21.15.7:0/266913671 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 -- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 msgr2=0x7fa9dc0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 -- 172.21.15.7:0/266913671 >> v2:172.21.15.7:3300/0 conn(0x7fa9dc0c88e0 msgr2=0x7fa9dc06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/266913671 >> v2:172.21.15.7:3300/0 conn(0x7fa9dc0c88e0 0x7fa9dc06d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 -- 172.21.15.7:0/266913671 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc15ce20 con 0x7fa9dc0ade60 2023-10-06T11:23:41.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.555+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc159f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0c2840 tx=0x7fa9d8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd0103516707033 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9d3fff700 1 -- 172.21.15.7:0/266913671 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800e070 con 0x7fa9dc0ade60 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9d3fff700 1 -- 172.21.15.7:0/266913671 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d800b970 con 0x7fa9dc0ade60 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9d3fff700 1 -- 172.21.15.7:0/266913671 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80092e0 con 0x7fa9dc0ade60 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0ade60 msgr2=0x7fa9dc159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 --2- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc159f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0c2840 tx=0x7fa9d8009840 comp rx=0 tx=0).stop 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- 172.21.15.7:0/266913671 shutdown_connections 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 --2- 172.21.15.7:0/266913671 >> v2:172.21.15.7:3300/0 conn(0x7fa9dc0c88e0 0x7fa9dc06d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 --2- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 --2- 172.21.15.7:0/266913671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc159f50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- 172.21.15.7:0/266913671 >> 172.21.15.7:0/266913671 conn(0x7fa9dc067e30 msgr2=0x7fa9dc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- 172.21.15.7:0/266913671 shutdown_connections 2023-10-06T11:23:41.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- 172.21.15.7:0/266913671 wait complete. 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 Processor -- start 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.559+0000 7fa9e3449700 1 -- start start 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0c88e0 0x7fa9dc06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc06ae30 con 0x7fa9dc0ade60 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc06b320 con 0x7fa9dc0c88e0 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc06b040 con 0x7fa9dc0ae230 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35530/0 (socket says 172.21.15.7:35530) 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 -- 172.21.15.7:0/2858956784 learned_addr learned my addr 172.21.15.7:0/2858956784 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e19e6700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0c88e0 0x7fa9dc06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e09e4700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 -- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 msgr2=0x7fa9dc14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 -- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0c88e0 msgr2=0x7fa9dc06a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0c88e0 0x7fa9dc06a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 -- 172.21.15.7:0/2858956784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8009ae0 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e11e5700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc15db70 tx=0x7fa9d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80088a0 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8008a00 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc169630 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8008dc0 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc169bf0 con 0x7fa9dc0ade60 2023-10-06T11:23:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.563+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc169490 con 0x7fa9dc0ade60 2023-10-06T11:23:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.567+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa9d8003040 con 0x7fa9dc0ade60 2023-10-06T11:23:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.567+0000 7fa9d17fa700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 0x7fa9c0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.567+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fa9d807e610 con 0x7fa9dc0ade60 2023-10-06T11:23:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.567+0000 7fa9e09e4700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 0x7fa9c0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.567+0000 7fa9e09e4700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 0x7fa9c0062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0c5460 tx=0x7fa9cc008040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.571+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa9d804e890 con 0x7fa9dc0ade60 2023-10-06T11:23:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.727+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9dc169490 con 0x7fa9c0060670 2023-10-06T11:23:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.727+0000 7fa9d17fa700 1 -- 172.21.15.7:0/2858956784 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9dc169490 con 0x7fa9c0060670 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 msgr2=0x7fa9c0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 0x7fa9c0062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0c5460 tx=0x7fa9cc008040 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 msgr2=0x7fa9dc14af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc15db70 tx=0x7fa9d8018000 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 shutdown_connections 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fa9c0060670 0x7fa9c0062b20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9dc0c88e0 0x7fa9dc06a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fa9dc0ae230 0x7fa9dc14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 --2- 172.21.15.7:0/2858956784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9dc0ade60 0x7fa9dc14af00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 >> 172.21.15.7:0/2858956784 conn(0x7fa9dc067e30 msgr2=0x7fa9dc1548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 shutdown_connections 2023-10-06T11:23:41.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.731+0000 7fa9e3449700 1 -- 172.21.15.7:0/2858956784 wait complete. 2023-10-06T11:23:41.738 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T11:23:41.752 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T11:23:41.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:41.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.867+0000 7f2e9ac58700 1 Processor -- start 2023-10-06T11:23:41.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.867+0000 7f2e9ac58700 1 -- start start 2023-10-06T11:23:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e94161bf0 0x7f2e94163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e940cc150 con 0x7f2e94158fe0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e940cb540 con 0x7f2e940ce3f0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e940cb820 con 0x7f2e94161bf0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e989f4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e94158fe0 0x7f2e940caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43106/0 (socket says 172.21.15.7:43106) 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e989f4700 1 -- 172.21.15.7:0/2331896397 learned_addr learned my addr 172.21.15.7:0/2331896397 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e991f5700 1 --2- 172.21.15.7:0/2331896397 >> v2:172.21.15.50:3300/0 conn(0x7f2e94161bf0 0x7f2e94163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679115725 0 0) 0x7f2e940cc150 con 0x7f2e94158fe0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003570 con 0x7f2e94158fe0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104981516 0 0) 0x7f2e7c003570 con 0x7f2e94158fe0 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e991f5700 1 -- 172.21.15.7:0/2331896397 >> v1:172.21.15.50:6789/0 conn(0x7f2e940ce3f0 legacy=0x7f2e940ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e991f5700 1 -- 172.21.15.7:0/2331896397 >> v1:172.21.15.7:6789/0 conn(0x7f2e94158fe0 legacy=0x7f2e940caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e991f5700 1 -- 172.21.15.7:0/2331896397 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940cbb00 con 0x7f2e94161bf0 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e991f5700 1 --2- 172.21.15.7:0/2331896397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e94163fd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009e60 tx=0x7f2e8c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b752dfce667e266d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00e070 con 0x7f2e94161bf0 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c00b850 con 0x7f2e94161bf0 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e8b7fe700 1 -- 172.21.15.7:0/2331896397 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90001470 con 0x7f2e94161bf0 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- 172.21.15.7:0/2331896397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 msgr2=0x7f2e94163fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 --2- 172.21.15.7:0/2331896397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e94163fd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009e60 tx=0x7f2e8c009780 comp rx=0 tx=0).stop 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- 172.21.15.7:0/2331896397 shutdown_connections 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 --2- 172.21.15.7:0/2331896397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e94163fd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- 172.21.15.7:0/2331896397 >> 172.21.15.7:0/2331896397 conn(0x7f2e940688c0 msgr2=0x7f2e940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- 172.21.15.7:0/2331896397 shutdown_connections 2023-10-06T11:23:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.871+0000 7f2e9ac58700 1 -- 172.21.15.7:0/2331896397 wait complete. 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 Processor -- start 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 -- start start 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 0x7f2e940c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e94158fe0 0x7f2e940c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e941593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94152ca0 con 0x7f2e940ce3f0 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940c8400 con 0x7f2e94161bf0 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e9ac58700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940c7ad0 con 0x7f2e94158fe0 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e989f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 0x7f2e940c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e991f5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e941593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e989f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 0x7f2e940c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35534/0 (socket says 172.21.15.7:35534) 2023-10-06T11:23:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e991f5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e941593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:58356/0 (socket says 172.21.15.7:58356) 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e989f4700 1 -- 172.21.15.7:0/3918231873 learned_addr learned my addr 172.21.15.7:0/3918231873 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.875+0000 7f2e8bfff700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e94158fe0 0x7f2e940c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 -- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e94158fe0 msgr2=0x7f2e940c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2e94158fe0 0x7f2e940c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 -- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 msgr2=0x7f2e941593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94161bf0 0x7f2e941593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c009a20 con 0x7f2e940ce3f0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e989f4700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 0x7f2e940c8890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cb90 tx=0x7f2e90001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90007150 con 0x7f2e940ce3f0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e900072d0 con 0x7f2e940ce3f0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e940c7db0 con 0x7f2e940ce3f0 2023-10-06T11:23:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90007690 con 0x7f2e940ce3f0 2023-10-06T11:23:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e941598f0 con 0x7f2e940ce3f0 2023-10-06T11:23:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e9404be00 con 0x7f2e940ce3f0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.879+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2e90004a40 con 0x7f2e940ce3f0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e80060670 0x7f2e80062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f2e9007c270 con 0x7f2e940ce3f0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e8bfff700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e80060670 0x7f2e80062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7f2e80067500 0x7f2e80069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e8006a000 con 0x7f2e80067500 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9007e7d0 con 0x7f2e940ce3f0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e991f5700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7f2e80067500 0x7f2e80069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e8bfff700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e80060670 0x7f2e80062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2e9416c3b0 tx=0x7f2e84007ec0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e991f5700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7f2e80067500 0x7f2e80069930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.883+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== osd.2 v2:172.21.15.7:6824/33406 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2e8006a000 con 0x7f2e80067500 2023-10-06T11:23:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 --> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2e94068c90 con 0x7f2e80067500 2023-10-06T11:23:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e88ff9700 1 -- 172.21.15.7:0/3918231873 <== osd.2 v2:172.21.15.7:6824/33406 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2e94068c90 con 0x7f2e80067500 2023-10-06T11:23:41.908 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:41.909 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:41.909 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7f2e80067500 msgr2=0x7f2e80069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:6824/33406,v1:172.21.15.7:6825/33406] conn(0x7f2e80067500 0x7f2e80069930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e80060670 msgr2=0x7f2e80062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2e80060670 0x7f2e80062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2e9416c3b0 tx=0x7f2e84007ec0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 msgr2=0x7f2e940c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e9ac58700 1 --2- 172.21.15.7:0/3918231873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e940ce3f0 0x7f2e940c8890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cb90 tx=0x7f2e90001ad0 comp rx=0 tx=0).stop 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.903+0000 7f2e991f5700 1 -- 172.21.15.7:0/3918231873 reap_dead start 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.907+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 shutdown_connections 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.907+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 >> 172.21.15.7:0/3918231873 conn(0x7f2e940688c0 msgr2=0x7f2e9406aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.907+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 shutdown_connections 2023-10-06T11:23:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:41.907+0000 7f2e9ac58700 1 -- 172.21.15.7:0/3918231873 wait complete. 2023-10-06T11:23:41.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T11:23:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.031+0000 7f3438ec6700 1 Processor -- start 2023-10-06T11:23:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.031+0000 7f3438ec6700 1 -- start start 2023-10-06T11:23:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f343415d440 0x7f343406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343406ac30 con 0x7f343415d070 2023-10-06T11:23:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34340ca0c0 con 0x7f34340c83a0 2023-10-06T11:23:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343406a020 con 0x7f343415d440 2023-10-06T11:23:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f343415d070 0x7f34341618a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54418/0 (socket says 172.21.15.7:54418) 2023-10-06T11:23:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343259c700 1 -- 172.21.15.7:0/1196972282 learned_addr learned my addr 172.21.15.7:0/1196972282 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3431d9b700 1 --2- 172.21.15.7:0/1196972282 >> v2:172.21.15.50:3300/0 conn(0x7f343415d440 0x7f343406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343159a700 1 -- 172.21.15.7:0/1196972282 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796459476 0 0) 0x7f343406ac30 con 0x7f343415d070 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343159a700 1 -- 172.21.15.7:0/1196972282 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3420003570 con 0x7f343415d070 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3431d9b700 1 -- 172.21.15.7:0/1196972282 >> v1:172.21.15.50:6789/0 conn(0x7f34340c83a0 legacy=0x7f34340c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3431d9b700 1 -- 172.21.15.7:0/1196972282 >> v1:172.21.15.7:6789/0 conn(0x7f343415d070 legacy=0x7f34341618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3431d9b700 1 -- 172.21.15.7:0/1196972282 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343406a300 con 0x7f343415d440 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3431d9b700 1 --2- 172.21.15.7:0/1196972282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f343415d440 0x7f343406b0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34240098e0 tx=0x7f34240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0dee91451b7f21b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343159a700 1 -- 172.21.15.7:0/1196972282 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400b640 con 0x7f343415d440 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343159a700 1 -- 172.21.15.7:0/1196972282 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400b7a0 con 0x7f343415d440 2023-10-06T11:23:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f343159a700 1 -- 172.21.15.7:0/1196972282 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400bb20 con 0x7f343415d440 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- 172.21.15.7:0/1196972282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f343415d440 msgr2=0x7f343406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 --2- 172.21.15.7:0/1196972282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f343415d440 0x7f343406b0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34240098e0 tx=0x7f34240091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- 172.21.15.7:0/1196972282 shutdown_connections 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 --2- 172.21.15.7:0/1196972282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f343415d440 0x7f343406b0c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- 172.21.15.7:0/1196972282 >> 172.21.15.7:0/1196972282 conn(0x7f3434067e30 msgr2=0x7f3434068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- 172.21.15.7:0/1196972282 shutdown_connections 2023-10-06T11:23:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.035+0000 7f3438ec6700 1 -- 172.21.15.7:0/1196972282 wait complete. 2023-10-06T11:23:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 Processor -- start 2023-10-06T11:23:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 -- start start 2023-10-06T11:23:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34340c83a0 0x7f34340b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343415d070 0x7f34340c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34340b1660 con 0x7f343415d440 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434170fd0 con 0x7f34340c83a0 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3438ec6700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34341712b0 con 0x7f343415d070 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59464/0 (socket says 172.21.15.7:59464) 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f343259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34340c83a0 0x7f34340b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 -- 172.21.15.7:0/874697430 learned_addr learned my addr 172.21.15.7:0/874697430 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3431d9b700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343415d070 0x7f34340c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 -- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343415d070 msgr2=0x7f34340c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343415d070 0x7f34340c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 -- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34340c83a0 msgr2=0x7f34340b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34340c83a0 0x7f34340b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 -- 172.21.15.7:0/874697430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009420 con 0x7f343415d440 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f3432d9d700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c8e0 tx=0x7f342c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c012630 con 0x7f343415d440 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.039+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c012790 con 0x7f343415d440 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3434171590 con 0x7f343415d440 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c012b50 con 0x7f343415d440 2023-10-06T11:23:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3434171d90 con 0x7f343415d440 2023-10-06T11:23:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3434070f20 con 0x7f343415d440 2023-10-06T11:23:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f342c01a3a0 con 0x7f343415d440 2023-10-06T11:23:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f342a7fc700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 0x7f3414062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f342c07cc80 con 0x7f343415d440 2023-10-06T11:23:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.043+0000 7f343259c700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 0x7f3414062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.047+0000 7f343259c700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 0x7f3414062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f341c007fd0 tx=0x7f341c009040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.055+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f342c050a20 con 0x7f343415d440 2023-10-06T11:23:42.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f343406a5e0 con 0x7f3414060670 2023-10-06T11:23:42.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f342a7fc700 1 -- 172.21.15.7:0/874697430 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f343406a5e0 con 0x7f3414060670 2023-10-06T11:23:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 msgr2=0x7f3414062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 0x7f3414062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f341c007fd0 tx=0x7f341c009040 comp rx=0 tx=0).stop 2023-10-06T11:23:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 msgr2=0x7f3434151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c8e0 tx=0x7f342c001710 comp rx=0 tx=0).stop 2023-10-06T11:23:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.215+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 shutdown_connections 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f3414060670 0x7f3414062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343415d440 0x7f3434151c90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f343415d070 0x7f34340c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 --2- 172.21.15.7:0/874697430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34340c83a0 0x7f34340b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 >> 172.21.15.7:0/874697430 conn(0x7f3434067e30 msgr2=0x7f343415c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 shutdown_connections 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.219+0000 7f3438ec6700 1 -- 172.21.15.7:0/874697430 wait complete. 2023-10-06T11:23:42.222 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T11:23:42.233 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T11:23:42.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.343+0000 7f0a7f26e700 1 Processor -- start 2023-10-06T11:23:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 -- start start 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a78158ea0 0x7f0a780ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0a780ccd70 0x7f0a780b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a780cac70 con 0x7f0a7806d940 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a78068d40 con 0x7f0a780ccd70 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7f26e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780caea0 con 0x7f0a78158ea0 2023-10-06T11:23:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7da6b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0a7806d940 0x7f0a780cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54422/0 (socket says 172.21.15.7:54422) 2023-10-06T11:23:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7da6b700 1 -- 172.21.15.7:0/1099145580 learned_addr learned my addr 172.21.15.7:0/1099145580 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/1099145580 >> v2:172.21.15.50:3300/0 conn(0x7f0a780ccd70 0x7f0a780b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7e26c700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a78158ea0 0x7f0a780ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260747608 0 0) 0x7f0a780cac70 con 0x7f0a7806d940 2023-10-06T11:23:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64003570 con 0x7f0a7806d940 2023-10-06T11:23:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a78158ea0 msgr2=0x7f0a780ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a78158ea0 0x7f0a780ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/1099145580 >> v1:172.21.15.7:6789/0 conn(0x7f0a7806d940 legacy=0x7f0a780cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462540474 0 0) 0x7f0a64003570 con 0x7f0a7806d940 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/1099145580 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a780cb180 con 0x7f0a780ccd70 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a780ccd70 0x7f0a780b0490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009e60 tx=0x7f0a70009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9dc0e4895285d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000e070 con 0x7f0a780ccd70 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7000b7a0 con 0x7f0a780ccd70 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.347+0000 7f0a7d26a700 1 -- 172.21.15.7:0/1099145580 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c001470 con 0x7f0a780ccd70 2023-10-06T11:23:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a780ccd70 msgr2=0x7f0a780b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a780ccd70 0x7f0a780b0490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009e60 tx=0x7f0a70009780 comp rx=0 tx=0).stop 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- 172.21.15.7:0/1099145580 shutdown_connections 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a780ccd70 0x7f0a780b0490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/1099145580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a78158ea0 0x7f0a780ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- 172.21.15.7:0/1099145580 >> 172.21.15.7:0/1099145580 conn(0x7f0a78067e30 msgr2=0x7f0a78068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- 172.21.15.7:0/1099145580 shutdown_connections 2023-10-06T11:23:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- 172.21.15.7:0/1099145580 wait complete. 2023-10-06T11:23:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 Processor -- start 2023-10-06T11:23:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.351+0000 7f0a7f26e700 1 -- start start 2023-10-06T11:23:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7806d940 0x7f0a7815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a780ccd70 0x7f0a7814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 0x7f0a780c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780b2d50 con 0x7f0a78158ea0 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780c7f20 con 0x7f0a7806d940 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780b2fa0 con 0x7f0a780ccd70 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 0x7f0a780c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7da6b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a780ccd70 0x7f0a7814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7da6b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a780ccd70 0x7f0a7814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3301/0 says I am v2:172.21.15.7:39772/0 (socket says 172.21.15.7:39772) 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7da6b700 1 -- 172.21.15.7:0/2871443471 learned_addr learned my addr 172.21.15.7:0/2871443471 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 0x7f0a780c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59466/0 (socket says 172.21.15.7:59466) 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7e26c700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7806d940 0x7f0a7815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a780ccd70 msgr2=0x7f0a7814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f0a780ccd70 0x7f0a7814e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7806d940 msgr2=0x7f0a7815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a7806d940 0x7f0a7815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70009a20 con 0x7f0a78158ea0 2023-10-06T11:23:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7ea6d700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 0x7f0a780c8b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0a70015000 tx=0x7f0a70018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70012bc0 con 0x7f0a78158ea0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70012d60 con 0x7f0a78158ea0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a780b3280 con 0x7f0a78158ea0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7001d680 con 0x7f0a78158ea0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a780b3b20 con 0x7f0a78158ea0 2023-10-06T11:23:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.355+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a7806c5c0 con 0x7f0a78158ea0 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0a70024070 con 0x7f0a78158ea0 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a5c060670 0x7f0a5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f0a7007ebb0 con 0x7f0a78158ea0 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f0a5c067500 0x7f0a5c069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a5c06a000 con 0x7f0a5c067500 2023-10-06T11:23:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a70083240 con 0x7f0a78158ea0 2023-10-06T11:23:42.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a7e26c700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a5c060670 0x7f0a5c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a7da6b700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f0a5c067500 0x7f0a5c069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a7da6b700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f0a5c067500 0x7f0a5c069930 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a7e26c700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a5c060670 0x7f0a5c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009d10 tx=0x7f0a740091a0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.359+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== osd.3 v2:172.21.15.7:6800/33384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0a5c06a000 con 0x7f0a5c067500 2023-10-06T11:23:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 --> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a780cb460 con 0x7f0a5c067500 2023-10-06T11:23:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a6a7fc700 1 -- 172.21.15.7:0/2871443471 <== osd.3 v2:172.21.15.7:6800/33384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0a780cb460 con 0x7f0a5c067500 2023-10-06T11:23:42.379 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:42.379 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f0a5c067500 msgr2=0x7f0a5c069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:6800/33384,v1:172.21.15.7:6801/33384] conn(0x7f0a5c067500 0x7f0a5c069930 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a5c060670 msgr2=0x7f0a5c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f0a5c060670 0x7f0a5c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009d10 tx=0x7f0a740091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 msgr2=0x7f0a780c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 --2- 172.21.15.7:0/2871443471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a78158ea0 0x7f0a780c8b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0a70015000 tx=0x7f0a70018000 comp rx=0 tx=0).stop 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7ea6d700 1 -- 172.21.15.7:0/2871443471 reap_dead start 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 shutdown_connections 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 >> 172.21.15.7:0/2871443471 conn(0x7f0a78067e30 msgr2=0x7f0a78155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 shutdown_connections 2023-10-06T11:23:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.375+0000 7f0a7f26e700 1 -- 172.21.15.7:0/2871443471 wait complete. 2023-10-06T11:23:42.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T11:23:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 Processor -- start 2023-10-06T11:23:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- start start 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febd415ab80 0x7febd40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 0x7febd415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd415b3c0 con 0x7febd415af50 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd415f540 con 0x7febd415ab80 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd415e950 con 0x7febd40afa90 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febd415ab80 0x7febd40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febd415af50 0x7febd40af380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54426/0 (socket says 172.21.15.7:54426) 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd37fe700 1 -- 172.21.15.7:0/2266642551 learned_addr learned my addr 172.21.15.7:0/2266642551 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd8c84700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 0x7febd415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 -- 172.21.15.7:0/2266642551 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 msgr2=0x7febd415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 0x7febd415f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 -- 172.21.15.7:0/2266642551 >> v1:172.21.15.7:6789/0 conn(0x7febd415af50 legacy=0x7febd40af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 -- 172.21.15.7:0/2266642551 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd415ec10 con 0x7febd415ab80 2023-10-06T11:23:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd8c84700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 0x7febd415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd3fff700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415ab80 0x7febd40ae730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098e0 tx=0x7febbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bfa52c92eedb97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd2ffd700 1 -- 172.21.15.7:0/2266642551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc009e80 con 0x7febd415ab80 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd2ffd700 1 -- 172.21.15.7:0/2266642551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc00b860 con 0x7febd415ab80 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd2ffd700 1 -- 172.21.15.7:0/2266642551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00bbe0 con 0x7febd415ab80 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- 172.21.15.7:0/2266642551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415ab80 msgr2=0x7febd40ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415ab80 0x7febd40ae730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098e0 tx=0x7febbc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- 172.21.15.7:0/2266642551 shutdown_connections 2023-10-06T11:23:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd40afa90 0x7febd415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 --2- 172.21.15.7:0/2266642551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415ab80 0x7febd40ae730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- 172.21.15.7:0/2266642551 >> 172.21.15.7:0/2266642551 conn(0x7febd40688a0 msgr2=0x7febd40cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- 172.21.15.7:0/2266642551 shutdown_connections 2023-10-06T11:23:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.503+0000 7febd9485700 1 -- 172.21.15.7:0/2266642551 wait complete. 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 Processor -- start 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 -- start start 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd40afa90 0x7febd40a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd415ab80 0x7febd416e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd416e750 con 0x7febd415af50 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4157e10 con 0x7febd40afa90 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd9485700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40a58b0 con 0x7febd415ab80 2023-10-06T11:23:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd37fe700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd415ab80 0x7febd416e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59482/0 (socket says 172.21.15.7:59482) 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 -- 172.21.15.7:0/1155596151 learned_addr learned my addr 172.21.15.7:0/1155596151 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd3fff700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd40afa90 0x7febd40a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 -- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd415ab80 msgr2=0x7febd416e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd415ab80 0x7febd416e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 -- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd40afa90 msgr2=0x7febd40a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd40afa90 0x7febd40a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.507+0000 7febd8c84700 1 -- 172.21.15.7:0/1155596151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009420 con 0x7febd415af50 2023-10-06T11:23:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febd8c84700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7febc400e870 tx=0x7febc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4013410 con 0x7febd415af50 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4013570 con 0x7febd415af50 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40a4ca0 con 0x7febd415af50 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc40138f0 con 0x7febd415af50 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd40a5540 con 0x7febd415af50 2023-10-06T11:23:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40a50a0 con 0x7febd415af50 2023-10-06T11:23:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7febc4013ca0 con 0x7febd415af50 2023-10-06T11:23:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 0x7febb4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7febc407c800 con 0x7febd415af50 2023-10-06T11:23:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.511+0000 7febd3fff700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 0x7febb4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.515+0000 7febd3fff700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 0x7febb4062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febbc0091e0 tx=0x7febbc017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.519+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7febc4050490 con 0x7febd415af50 2023-10-06T11:23:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.683+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7febd415eef0 con 0x7febb4060670 2023-10-06T11:23:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.683+0000 7febcbfff700 1 -- 172.21.15.7:0/1155596151 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7febd415eef0 con 0x7febb4060670 2023-10-06T11:23:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 msgr2=0x7febb4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 0x7febb4062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febbc0091e0 tx=0x7febbc017000 comp rx=0 tx=0).stop 2023-10-06T11:23:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 msgr2=0x7febd40add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7febc400e870 tx=0x7febc4008040 comp rx=0 tx=0).stop 2023-10-06T11:23:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 shutdown_connections 2023-10-06T11:23:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7febb4060670 0x7febb4062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd415af50 0x7febd40add10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7febd415ab80 0x7febd416e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 --2- 172.21.15.7:0/1155596151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febd40afa90 0x7febd40a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 >> 172.21.15.7:0/1155596151 conn(0x7febd40688a0 msgr2=0x7febd40695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 shutdown_connections 2023-10-06T11:23:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.687+0000 7febd9485700 1 -- 172.21.15.7:0/1155596151 wait complete. 2023-10-06T11:23:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T11:23:42.703 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T11:23:42.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 Processor -- start 2023-10-06T11:23:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 -- start start 2023-10-06T11:23:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde700cc1a0 0x7fde700add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fde700b09a0 0x7fde700b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700b1340 con 0x7fde700cc1a0 2023-10-06T11:23:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700cadf0 con 0x7fde700b09a0 2023-10-06T11:23:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.819+0000 7fde6ffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde700cb280 con 0x7fde701568b0 2023-10-06T11:23:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde700cc1a0 0x7fde700add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6f7fe700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fde700b09a0 0x7fde700b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde700cc1a0 0x7fde700add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59488/0 (socket says 172.21.15.7:59488) 2023-10-06T11:23:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 -- 172.21.15.7:0/1090332580 learned_addr learned my addr 172.21.15.7:0/1090332580 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6effd700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 -- 172.21.15.7:0/1090332580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 msgr2=0x7fde70156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 -- 172.21.15.7:0/1090332580 >> v2:172.21.15.50:3300/0 conn(0x7fde700b09a0 msgr2=0x7fde700b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/1090332580 >> v2:172.21.15.50:3300/0 conn(0x7fde700b09a0 0x7fde700b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 -- 172.21.15.7:0/1090332580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700c8370 con 0x7fde700cc1a0 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700add50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde700c8560 tx=0x7fde58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e6ef57cf67438 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6dffb700 1 -- 172.21.15.7:0/1090332580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011630 con 0x7fde700cc1a0 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6dffb700 1 -- 172.21.15.7:0/1090332580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58011790 con 0x7fde700cc1a0 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6dffb700 1 -- 172.21.15.7:0/1090332580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011b50 con 0x7fde700cc1a0 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- 172.21.15.7:0/1090332580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 msgr2=0x7fde700add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700add50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde700c8560 tx=0x7fde58009250 comp rx=0 tx=0).stop 2023-10-06T11:23:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- 172.21.15.7:0/1090332580 shutdown_connections 2023-10-06T11:23:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 --2- 172.21.15.7:0/1090332580 >> v2:172.21.15.50:3300/0 conn(0x7fde700b09a0 0x7fde700b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700add50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 --2- 172.21.15.7:0/1090332580 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- 172.21.15.7:0/1090332580 >> 172.21.15.7:0/1090332580 conn(0x7fde70068880 msgr2=0x7fde700cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- 172.21.15.7:0/1090332580 shutdown_connections 2023-10-06T11:23:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- 172.21.15.7:0/1090332580 wait complete. 2023-10-06T11:23:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 Processor -- start 2023-10-06T11:23:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.823+0000 7fde6ffff700 1 -- start start 2023-10-06T11:23:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde700b09a0 0x7fde701470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700aa260 con 0x7fde700cc1a0 2023-10-06T11:23:42.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70169c90 con 0x7fde700b09a0 2023-10-06T11:23:42.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde701693d0 con 0x7fde701568b0 2023-10-06T11:23:42.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59504/0 (socket says 172.21.15.7:59504) 2023-10-06T11:23:42.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6f7fe700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 -- 172.21.15.7:0/3982698740 learned_addr learned my addr 172.21.15.7:0/3982698740 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:42.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6effd700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde700b09a0 0x7fde701470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 -- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 msgr2=0x7fde70148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fde701568b0 0x7fde70148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 -- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde700b09a0 msgr2=0x7fde701470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde700b09a0 0x7fde701470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580094f0 con 0x7fde700cc1a0 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6e7fc700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700b03c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde580094c0 tx=0x7fde58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5801bd00 con 0x7fde700cc1a0 2023-10-06T11:23:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5801be80 con 0x7fde700cc1a0 2023-10-06T11:23:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde700a9f80 con 0x7fde700cc1a0 2023-10-06T11:23:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580256c0 con 0x7fde700cc1a0 2023-10-06T11:23:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde70169920 con 0x7fde700cc1a0 2023-10-06T11:23:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.827+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde7006c410 con 0x7fde700cc1a0 2023-10-06T11:23:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fde58022070 con 0x7fde700cc1a0 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fde54060670 0x7fde54062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fde5807e4c0 con 0x7fde700cc1a0 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde6effd700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fde54060670 0x7fde54062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7fde54067500 0x7fde54069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde5406a000 con 0x7fde54067500 2023-10-06T11:23:42.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde58080a20 con 0x7fde700cc1a0 2023-10-06T11:23:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde6f7fe700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7fde54067500 0x7fde54069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde6effd700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fde54060670 0x7fde54062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde60009db0 tx=0x7fde600091a0 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.831+0000 7fde6f7fe700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7fde54067500 0x7fde54069930 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.835+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== osd.4 v2:172.21.15.50:6800/33665 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fde5406a000 con 0x7fde54067500 2023-10-06T11:23:42.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.851+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 --> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fde7004b5c0 con 0x7fde54067500 2023-10-06T11:23:42.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.851+0000 7fde66ffd700 1 -- 172.21.15.7:0/3982698740 <== osd.4 v2:172.21.15.50:6800/33665 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fde7004b5c0 con 0x7fde54067500 2023-10-06T11:23:42.856 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:42.857 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:42.857 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7fde54067500 msgr2=0x7fde54069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6800/33665,v1:172.21.15.50:6801/33665] conn(0x7fde54067500 0x7fde54069930 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fde54060670 msgr2=0x7fde54062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fde54060670 0x7fde54062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde60009db0 tx=0x7fde600091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 msgr2=0x7fde700b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 --2- 172.21.15.7:0/3982698740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde700cc1a0 0x7fde700b03c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde580094c0 tx=0x7fde58016000 comp rx=0 tx=0).stop 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6f7fe700 1 -- 172.21.15.7:0/3982698740 reap_dead start 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 shutdown_connections 2023-10-06T11:23:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 >> 172.21.15.7:0/3982698740 conn(0x7fde70068880 msgr2=0x7fde7014b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 shutdown_connections 2023-10-06T11:23:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.855+0000 7fde6ffff700 1 -- 172.21.15.7:0/3982698740 wait complete. 2023-10-06T11:23:42.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T11:23:42.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.987+0000 7fd7c1028700 1 Processor -- start 2023-10-06T11:23:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.987+0000 7fd7c1028700 1 -- start start 2023-10-06T11:23:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd7bc15a8f0 0x7fd7bc0aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0c7470 con 0x7fd7bc151500 2023-10-06T11:23:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc1572f0 con 0x7fd7bc151130 2023-10-06T11:23:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc15ae40 con 0x7fd7bc15a8f0 2023-10-06T11:23:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7ba59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd7bc151500 0x7fd7bc0c6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54442/0 (socket says 172.21.15.7:54442) 2023-10-06T11:23:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7ba59c700 1 -- 172.21.15.7:0/1355823524 learned_addr learned my addr 172.21.15.7:0/1355823524 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:42.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7bb59e700 1 --2- 172.21.15.7:0/1355823524 >> v2:172.21.15.50:3300/0 conn(0x7fd7bc15a8f0 0x7fd7bc0aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:42.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548131909 0 0) 0x7fd7bc1572f0 con 0x7fd7bc151130 2023-10-06T11:23:42.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003570 con 0x7fd7bc151130 2023-10-06T11:23:42.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839444914 0 0) 0x7fd7bc0c7470 con 0x7fd7bc151500 2023-10-06T11:23:42.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc1572f0 con 0x7fd7bc151500 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7bb59e700 1 -- 172.21.15.7:0/1355823524 >> v1:172.21.15.50:6789/0 conn(0x7fd7bc151130 legacy=0x7fd7bc069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7bb59e700 1 -- 172.21.15.7:0/1355823524 >> v1:172.21.15.7:6789/0 conn(0x7fd7bc151500 legacy=0x7fd7bc0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1447322981 0 0) 0x7fd7bc1572f0 con 0x7fd7bc151500 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7bb59e700 1 -- 172.21.15.7:0/1355823524 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc15b120 con 0x7fd7bc15a8f0 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7bb59e700 1 --2- 172.21.15.7:0/1355823524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc15a8f0 0x7fd7bc0aa710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009910 tx=0x7fd7b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=926ee3379255764a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00013a0 con 0x7fd7bc15a8f0 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4011420 con 0x7fd7bc15a8f0 2023-10-06T11:23:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7b9d9b700 1 -- 172.21.15.7:0/1355823524 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40117c0 con 0x7fd7bc15a8f0 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- 172.21.15.7:0/1355823524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc15a8f0 msgr2=0x7fd7bc0aa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 --2- 172.21.15.7:0/1355823524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc15a8f0 0x7fd7bc0aa710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009910 tx=0x7fd7b40091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- 172.21.15.7:0/1355823524 shutdown_connections 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 --2- 172.21.15.7:0/1355823524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc15a8f0 0x7fd7bc0aa710 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.991+0000 7fd7c1028700 1 -- 172.21.15.7:0/1355823524 >> 172.21.15.7:0/1355823524 conn(0x7fd7bc067e30 msgr2=0x7fd7bc068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- 172.21.15.7:0/1355823524 shutdown_connections 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- 172.21.15.7:0/1355823524 wait complete. 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 Processor -- start 2023-10-06T11:23:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- start start 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 0x7fd7bc0c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 0x7fd7bc0c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc166e50 con 0x7fd7bc151500 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c3330 con 0x7fd7bc151130 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7c1028700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c3610 con 0x7fd7bc15a8f0 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59518/0 (socket says 172.21.15.7:59518) 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 -- 172.21.15.7:0/864121896 learned_addr learned my addr 172.21.15.7:0/864121896 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:43.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7bb59e700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 0x7fd7bc0c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7bad9d700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 0x7fd7bc0c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 -- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 msgr2=0x7fd7bc0c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 0x7fd7bc0c12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 -- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 msgr2=0x7fd7bc0c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 0x7fd7bc0c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7ba59c700 1 -- 172.21.15.7:0/864121896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009420 con 0x7fd7bc151500 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.995+0000 7fd7bb59e700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 0x7fd7bc0c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7bad9d700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 0x7fd7bc0c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7ba59c700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cb90 tx=0x7fd7b0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0007e90 con 0x7fd7bc151500 2023-10-06T11:23:43.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0007360 con 0x7fd7bc151500 2023-10-06T11:23:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc0c38f0 con 0x7fd7bc151500 2023-10-06T11:23:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0007700 con 0x7fd7bc151500 2023-10-06T11:23:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc0c3020 con 0x7fd7bc151500 2023-10-06T11:23:43.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc070f20 con 0x7fd7bc151500 2023-10-06T11:23:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd7b0004a40 con 0x7fd7bc151500 2023-10-06T11:23:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 0x7fd7a8062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fd7b007c460 con 0x7fd7bc151500 2023-10-06T11:23:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:42.999+0000 7fd7bad9d700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 0x7fd7a8062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.003+0000 7fd7bad9d700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 0x7fd7a8062bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac003fa0 tx=0x7fd7ac009040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.007+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7b004c750 con 0x7fd7bc151500 2023-10-06T11:23:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7bc0c2230 con 0x7fd7a8060720 2023-10-06T11:23:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7a2ffd700 1 -- 172.21.15.7:0/864121896 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7bc0c2230 con 0x7fd7a8060720 2023-10-06T11:23:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 msgr2=0x7fd7a8062bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 0x7fd7a8062bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac003fa0 tx=0x7fd7ac009040 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 msgr2=0x7fd7bc1504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cb90 tx=0x7fd7b0001b10 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 shutdown_connections 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fd7a8060720 0x7fd7a8062bd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fd7bc15a8f0 0x7fd7bc0c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7bc151500 0x7fd7bc1504f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.167+0000 7fd7c1028700 1 --2- 172.21.15.7:0/864121896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7bc151130 0x7fd7bc0c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.171+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 >> 172.21.15.7:0/864121896 conn(0x7fd7bc067e30 msgr2=0x7fd7bc0aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.171+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 shutdown_connections 2023-10-06T11:23:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.171+0000 7fd7c1028700 1 -- 172.21.15.7:0/864121896 wait complete. 2023-10-06T11:23:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T11:23:43.185 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T11:23:43.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.295+0000 7fdd77fff700 1 Processor -- start 2023-10-06T11:23:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.295+0000 7fdd77fff700 1 -- start start 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd78156d40 0x7fdd780ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdd780694e0 0x7fdd780cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd780681f0 con 0x7fdd780aec00 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd78068440 con 0x7fdd78156d40 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd78068720 con 0x7fdd780694e0 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd780aec00 0x7fdd780aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54454/0 (socket says 172.21.15.7:54454) 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd777fe700 1 -- 172.21.15.7:0/1984138825 learned_addr learned my addr 172.21.15.7:0/1984138825 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1984138825 >> v2:172.21.15.7:3300/0 conn(0x7fdd78156d40 0x7fdd780ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd6e7fc700 1 --2- 172.21.15.7:0/1984138825 >> v2:172.21.15.50:3300/0 conn(0x7fdd780694e0 0x7fdd780cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd767fc700 1 -- 172.21.15.7:0/1984138825 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751271307 0 0) 0x7fdd780681f0 con 0x7fdd780aec00 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd767fc700 1 -- 172.21.15.7:0/1984138825 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd64003570 con 0x7fdd780aec00 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1984138825 >> v2:172.21.15.50:3300/0 conn(0x7fdd780694e0 msgr2=0x7fdd780cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1984138825 >> v2:172.21.15.50:3300/0 conn(0x7fdd780694e0 0x7fdd780cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1984138825 >> v1:172.21.15.7:6789/0 conn(0x7fdd780aec00 legacy=0x7fdd780aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 -- 172.21.15.7:0/1984138825 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78068a00 con 0x7fdd78156d40 2023-10-06T11:23:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/1984138825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78156d40 0x7fdd780ccb60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b2a60 tx=0x7fdd680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3eb2b73d9401a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd767fc700 1 -- 172.21.15.7:0/1984138825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68011630 con 0x7fdd78156d40 2023-10-06T11:23:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd767fc700 1 -- 172.21.15.7:0/1984138825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680117b0 con 0x7fdd78156d40 2023-10-06T11:23:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd767fc700 1 -- 172.21.15.7:0/1984138825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68011b70 con 0x7fdd78156d40 2023-10-06T11:23:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- 172.21.15.7:0/1984138825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78156d40 msgr2=0x7fdd780ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 --2- 172.21.15.7:0/1984138825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78156d40 0x7fdd780ccb60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b2a60 tx=0x7fdd680091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- 172.21.15.7:0/1984138825 shutdown_connections 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 --2- 172.21.15.7:0/1984138825 >> v2:172.21.15.50:3300/0 conn(0x7fdd780694e0 0x7fdd780cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 --2- 172.21.15.7:0/1984138825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd78156d40 0x7fdd780ccb60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- 172.21.15.7:0/1984138825 >> 172.21.15.7:0/1984138825 conn(0x7fdd7806ea80 msgr2=0x7fdd7806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- 172.21.15.7:0/1984138825 shutdown_connections 2023-10-06T11:23:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.299+0000 7fdd77fff700 1 -- 172.21.15.7:0/1984138825 wait complete. 2023-10-06T11:23:43.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 Processor -- start 2023-10-06T11:23:43.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 -- start start 2023-10-06T11:23:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdd780694e0 0x7fdd7814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 0x7fdd7814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78156d40 0x7fdd7814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c6270 con 0x7fdd780aec00 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7814d2d0 con 0x7fdd78156d40 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd77fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7814c330 con 0x7fdd780694e0 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 0x7fdd7814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdd780694e0 0x7fdd7814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 0x7fdd7814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59534/0 (socket says 172.21.15.7:59534) 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd777fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78156d40 0x7fdd7814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.303+0000 7fdd6e7fc700 1 -- 172.21.15.7:0/4030791773 learned_addr learned my addr 172.21.15.7:0/4030791773 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 -- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdd780694e0 msgr2=0x7fdd7814bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fdd780694e0 0x7fdd7814bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 -- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78156d40 msgr2=0x7fdd7814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78156d40 0x7fdd7814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009420 con 0x7fdd780aec00 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd777fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78156d40 0x7fdd7814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6e7fc700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 0x7fdd7814c4e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ecb0 tx=0x7fdd600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6000c850 con 0x7fdd780aec00 2023-10-06T11:23:43.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6000c9b0 con 0x7fdd780aec00 2023-10-06T11:23:43.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd780c4730 con 0x7fdd780aec00 2023-10-06T11:23:43.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6000cd70 con 0x7fdd780aec00 2023-10-06T11:23:43.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd780c6480 con 0x7fdd780aec00 2023-10-06T11:23:43.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd7806d720 con 0x7fdd780aec00 2023-10-06T11:23:43.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.307+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fdd60013680 con 0x7fdd780aec00 2023-10-06T11:23:43.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdd58060670 0x7fdd58062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fdd6007d530 con 0x7fdd780aec00 2023-10-06T11:23:43.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fdd58067500 0x7fdd58069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdd58060670 0x7fdd58062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd5806a000 con 0x7fdd58067500 2023-10-06T11:23:43.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6007d900 con 0x7fdd780aec00 2023-10-06T11:23:43.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd777fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fdd58067500 0x7fdd58069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd76ffd700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdd58060670 0x7fdd58062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b2a60 tx=0x7fdd68017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd777fe700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fdd58067500 0x7fdd58069930 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.311+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== osd.5 v2:172.21.15.50:6816/33666 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdd5806a000 con 0x7fdd58067500 2023-10-06T11:23:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 --> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdd780c6820 con 0x7fdd58067500 2023-10-06T11:23:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd6f7fe700 1 -- 172.21.15.7:0/4030791773 <== osd.5 v2:172.21.15.50:6816/33666 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdd780c6820 con 0x7fdd58067500 2023-10-06T11:23:43.335 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:43.335 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fdd58067500 msgr2=0x7fdd58069930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6816/33666,v1:172.21.15.50:6818/33666] conn(0x7fdd58067500 0x7fdd58069930 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdd58060670 msgr2=0x7fdd58062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fdd58060670 0x7fdd58062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b2a60 tx=0x7fdd68017000 comp rx=0 tx=0).stop 2023-10-06T11:23:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 msgr2=0x7fdd7814c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 --2- 172.21.15.7:0/4030791773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd780aec00 0x7fdd7814c4e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ecb0 tx=0x7fdd600091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd777fe700 1 -- 172.21.15.7:0/4030791773 reap_dead start 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 shutdown_connections 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 >> 172.21.15.7:0/4030791773 conn(0x7fdd7806ea80 msgr2=0x7fdd780ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 shutdown_connections 2023-10-06T11:23:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.331+0000 7fdd77fff700 1 -- 172.21.15.7:0/4030791773 wait complete. 2023-10-06T11:23:43.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T11:23:43.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 Processor -- start 2023-10-06T11:23:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 -- start start 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 0x7f4a4c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0ae860 con 0x7f4a4c0cc190 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0aeb40 con 0x7f4a4c0b07f0 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a51d3d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0aee20 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a4bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4a4c0cc190 0x7f4a4c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54466/0 (socket says 172.21.15.7:54466) 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.467+0000 7f4a4bfff700 1 -- 172.21.15.7:0/2039798594 learned_addr learned my addr 172.21.15.7:0/2039798594 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a50d3b700 1 --2- 172.21.15.7:0/2039798594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 0x7f4a4c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837604283 0 0) 0x7f4a4c0ae860 con 0x7f4a4c0cc190 2023-10-06T11:23:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a34003570 con 0x7f4a4c0cc190 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136310475 0 0) 0x7f4a4c0aeb40 con 0x7f4a4c0b07f0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a4c0ae860 con 0x7f4a4c0b07f0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a50d3b700 1 -- 172.21.15.7:0/2039798594 >> v1:172.21.15.50:6789/0 conn(0x7f4a4c0b07f0 legacy=0x7f4a4c0aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a50d3b700 1 -- 172.21.15.7:0/2039798594 >> v1:172.21.15.7:6789/0 conn(0x7f4a4c0cc190 legacy=0x7f4a4c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a50d3b700 1 -- 172.21.15.7:0/2039798594 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0af100 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a50d3b700 1 --2- 172.21.15.7:0/2039798594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 0x7f4a4c1501c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009910 tx=0x7f4a400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa84407af76e108 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011630 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40011790 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/2039798594 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011b10 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 -- 172.21.15.7:0/2039798594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 msgr2=0x7f4a4c1501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/2039798594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 0x7f4a4c1501c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009910 tx=0x7f4a400091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 -- 172.21.15.7:0/2039798594 shutdown_connections 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/2039798594 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c14fdf0 0x7f4a4c1501c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 -- 172.21.15.7:0/2039798594 >> 172.21.15.7:0/2039798594 conn(0x7f4a4c0688a0 msgr2=0x7f4a4c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 -- 172.21.15.7:0/2039798594 shutdown_connections 2023-10-06T11:23:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.471+0000 7f4a51d3d700 1 -- 172.21.15.7:0/2039798594 wait complete. 2023-10-06T11:23:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 Processor -- start 2023-10-06T11:23:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 -- start start 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c0b07f0 0x7f4a4c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c0cc190 0x7f4a4c16af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15c490 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c16aab0 con 0x7f4a4c0cc190 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a51d3d700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c14ecf0 con 0x7f4a4c0b07f0 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59536/0 (socket says 172.21.15.7:59536) 2023-10-06T11:23:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 -- 172.21.15.7:0/753667507 learned_addr learned my addr 172.21.15.7:0/753667507 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a50d3b700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c0b07f0 0x7f4a4c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a4bfff700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c0cc190 0x7f4a4c16af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 -- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c0b07f0 msgr2=0x7f4a4c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c0b07f0 0x7f4a4c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 -- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c0cc190 msgr2=0x7f4a4c16af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c0cc190 0x7f4a4c16af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.475+0000 7f4a5153c700 1 -- 172.21.15.7:0/753667507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009420 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a5153c700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c840 tx=0x7f4a44002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44002030 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440021b0 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44012630 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c16a780 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c15c1b0 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c16a5e0 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.479+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4a44012a20 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.483+0000 7f4a48ff9700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 0x7f4a38062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.483+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f4a4407c9e0 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.483+0000 7f4a50d3b700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 0x7f4a38062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.483+0000 7f4a50d3b700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 0x7f4a38062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009860 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.487+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4a4c16a5e0 con 0x7f4a4c14fdf0 2023-10-06T11:23:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.651+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a4c16a5e0 con 0x7f4a38060670 2023-10-06T11:23:43.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.655+0000 7f4a48ff9700 1 -- 172.21.15.7:0/753667507 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4a4c16a5e0 con 0x7f4a38060670 2023-10-06T11:23:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 msgr2=0x7f4a38062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 0x7f4a38062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009860 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2023-10-06T11:23:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 msgr2=0x7f4a4c141260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c840 tx=0x7f4a44002680 comp rx=0 tx=0).stop 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 shutdown_connections 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f4a38060670 0x7f4a38062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c14fdf0 0x7f4a4c141260 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c0cc190 0x7f4a4c16af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 --2- 172.21.15.7:0/753667507 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f4a4c0b07f0 0x7f4a4c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 >> 172.21.15.7:0/753667507 conn(0x7f4a4c0688a0 msgr2=0x7f4a4c14f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 shutdown_connections 2023-10-06T11:23:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.659+0000 7f4a51d3d700 1 -- 172.21.15.7:0/753667507 wait complete. 2023-10-06T11:23:43.664 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T11:23:43.679 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T11:23:43.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:43.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.787+0000 7f279df3f700 1 Processor -- start 2023-10-06T11:23:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.787+0000 7f279df3f700 1 -- start start 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 0x7f27980c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980c9b90 con 0x7f279806d6e0 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980ca020 con 0x7f27981593c0 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c8f80 con 0x7f2798158ff0 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279d73e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f279806d6e0 0x7f2798162d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54472/0 (socket says 172.21.15.7:54472) 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279d73e700 1 -- 172.21.15.7:0/1769496966 learned_addr learned my addr 172.21.15.7:0/1769496966 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279cf3d700 1 --2- 172.21.15.7:0/1769496966 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 0x7f27980c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047358419 0 0) 0x7f27980c9b90 con 0x7f279806d6e0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2780003570 con 0x7f279806d6e0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508234023 0 0) 0x7f27980ca020 con 0x7f27981593c0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27980c9b90 con 0x7f27981593c0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279cf3d700 1 -- 172.21.15.7:0/1769496966 >> v1:172.21.15.50:6789/0 conn(0x7f27981593c0 legacy=0x7f279806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279cf3d700 1 -- 172.21.15.7:0/1769496966 >> v1:172.21.15.7:6789/0 conn(0x7f279806d6e0 legacy=0x7f2798162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279cf3d700 1 -- 172.21.15.7:0/1769496966 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980c9260 con 0x7f2798158ff0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279cf3d700 1 --2- 172.21.15.7:0/1769496966 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 0x7f27980c8670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f278c009910 tx=0x7f278c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487b2a1dfa28d79e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c011630 con 0x7f2798158ff0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c011790 con 0x7f2798158ff0 2023-10-06T11:23:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f27977fe700 1 -- 172.21.15.7:0/1769496966 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c011b10 con 0x7f2798158ff0 2023-10-06T11:23:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 -- 172.21.15.7:0/1769496966 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 msgr2=0x7f27980c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.791+0000 7f279df3f700 1 --2- 172.21.15.7:0/1769496966 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 0x7f27980c8670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f278c009910 tx=0x7f278c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 -- 172.21.15.7:0/1769496966 shutdown_connections 2023-10-06T11:23:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 --2- 172.21.15.7:0/1769496966 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f2798158ff0 0x7f27980c8670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 -- 172.21.15.7:0/1769496966 >> 172.21.15.7:0/1769496966 conn(0x7f279806c7d0 msgr2=0x7f27980cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 -- 172.21.15.7:0/1769496966 shutdown_connections 2023-10-06T11:23:43.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 -- 172.21.15.7:0/1769496966 wait complete. 2023-10-06T11:23:43.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 Processor -- start 2023-10-06T11:23:43.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.795+0000 7f279df3f700 1 -- start start 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 0x7f2798151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2798158ff0 0x7f27980b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f27981593c0 0x7f27980b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b2970 con 0x7f2798158ff0 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c7350 con 0x7f279806d6e0 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b2b60 con 0x7f27981593c0 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f2797fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2798158ff0 0x7f27980b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 0x7f2798151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 0x7f2798151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:56276/0 (socket says 172.21.15.7:56276) 2023-10-06T11:23:43.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 -- 172.21.15.7:0/1205834702 learned_addr learned my addr 172.21.15.7:0/1205834702 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279d73e700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f27981593c0 0x7f27980b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 -- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f27981593c0 msgr2=0x7f27980b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f27981593c0 0x7f27980b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 -- 172.21.15.7:0/1205834702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2798158ff0 msgr2=0x7f27980b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2798158ff0 0x7f27980b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c009420 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279cf3d700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 0x7f2798151830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f278c009f00 tx=0x7f278c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c011900 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c011a60 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980b2e40 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c025640 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.799+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2798151d70 con 0x7f279806d6e0 2023-10-06T11:23:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f279804b5c0 con 0x7f279806d6e0 2023-10-06T11:23:43.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f278c025a30 con 0x7f279806d6e0 2023-10-06T11:23:43.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2784060670 0x7f2784062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f278c07ee40 con 0x7f279806d6e0 2023-10-06T11:23:43.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f2784067480 0x7f27840698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2784069f80 con 0x7f2784067480 2023-10-06T11:23:43.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2797fff700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2784060670 0x7f2784062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f278c07f210 con 0x7f279806d6e0 2023-10-06T11:23:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f279d73e700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f2784067480 0x7f27840698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f2797fff700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2784060670 0x7f2784062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f279815e5d0 tx=0x7f2788007d30 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.803+0000 7f279d73e700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f2784067480 0x7f27840698b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.807+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== osd.6 v2:172.21.15.50:6808/33664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2784069f80 con 0x7f2784067480 2023-10-06T11:23:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.823+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 --> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f27980639c0 con 0x7f2784067480 2023-10-06T11:23:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f2794ff9700 1 -- 172.21.15.7:0/1205834702 <== osd.6 v2:172.21.15.50:6808/33664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f27980639c0 con 0x7f2784067480 2023-10-06T11:23:43.829 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:43.830 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:43.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f2784067480 msgr2=0x7f27840698b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6808/33664,v1:172.21.15.50:6809/33664] conn(0x7f2784067480 0x7f27840698b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2784060670 msgr2=0x7f2784062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f2784060670 0x7f2784062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f279815e5d0 tx=0x7f2788007d30 comp rx=0 tx=0).stop 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 msgr2=0x7f2798151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 --2- 172.21.15.7:0/1205834702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f279806d6e0 0x7f2798151830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f278c009f00 tx=0x7f278c017000 comp rx=0 tx=0).stop 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279d73e700 1 -- 172.21.15.7:0/1205834702 reap_dead start 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 shutdown_connections 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 >> 172.21.15.7:0/1205834702 conn(0x7f279806c7d0 msgr2=0x7f27980cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 shutdown_connections 2023-10-06T11:23:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.827+0000 7f279df3f700 1 -- 172.21.15.7:0/1205834702 wait complete. 2023-10-06T11:23:43.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T11:23:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.951+0000 7efca9602700 1 Processor -- start 2023-10-06T11:23:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.951+0000 7efca9602700 1 -- start start 2023-10-06T11:23:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 0x7efca40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca40cae00 con 0x7efca40ca470 2023-10-06T11:23:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40ae2d0 con 0x7efca40ade50 2023-10-06T11:23:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca40b0410 con 0x7efca40afb00 2023-10-06T11:23:43.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca8e01700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 0x7efca40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca37fe700 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7efca40ca470 0x7efca40ad740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.7:60366/0 (socket says 172.21.15.7:60366) 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca37fe700 1 -- 172.21.15.7:0/2396325691 learned_addr learned my addr 172.21.15.7:0/2396325691 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 -- 172.21.15.7:0/2396325691 >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 msgr2=0x7efca40ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 --2- 172.21.15.7:0/2396325691 >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 0x7efca40ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 -- 172.21.15.7:0/2396325691 >> v1:172.21.15.50:6789/0 conn(0x7efca40ca470 legacy=0x7efca40ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca2ffd700 1 -- 172.21.15.7:0/2396325691 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057133293 0 0) 0x7efca40cae00 con 0x7efca40ca470 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 -- 172.21.15.7:0/2396325691 --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40ca0a0 con 0x7efca40afb00 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca8e01700 1 --2- 172.21.15.7:0/2396325691 >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 0x7efca40ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:23:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca3fff700 1 --2- 172.21.15.7:0/2396325691 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40afed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc940098e0 tx=0x7efc940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb1db4b38a48aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca2ffd700 1 -- 172.21.15.7:0/2396325691 <== mon.2 v2:172.21.15.50:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9400b640 con 0x7efca40afb00 2023-10-06T11:23:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca2ffd700 1 -- 172.21.15.7:0/2396325691 <== mon.2 v2:172.21.15.50:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9400b7a0 con 0x7efca40afb00 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca2ffd700 1 -- 172.21.15.7:0/2396325691 <== mon.2 v2:172.21.15.50:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9400bb40 con 0x7efca40afb00 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- 172.21.15.7:0/2396325691 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 msgr2=0x7efca40afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 --2- 172.21.15.7:0/2396325691 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40afed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc940098e0 tx=0x7efc940091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- 172.21.15.7:0/2396325691 shutdown_connections 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 --2- 172.21.15.7:0/2396325691 >> v2:172.21.15.50:3300/0 conn(0x7efca40ade50 0x7efca40ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 --2- 172.21.15.7:0/2396325691 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40afed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- 172.21.15.7:0/2396325691 >> 172.21.15.7:0/2396325691 conn(0x7efca406bd70 msgr2=0x7efca406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- 172.21.15.7:0/2396325691 shutdown_connections 2023-10-06T11:23:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.955+0000 7efca9602700 1 -- 172.21.15.7:0/2396325691 wait complete. 2023-10-06T11:23:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 Processor -- start 2023-10-06T11:23:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- start start 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 0x7efca40aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 0x7efca41693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40c2500 con 0x7efca40ca470 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40ab610 con 0x7efca40ade50 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca40ab8f0 con 0x7efca40afb00 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 0x7efca40aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 0x7efca41693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 0x7efca40aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:56280/0 (socket says 172.21.15.7:56280) 2023-10-06T11:23:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca3fff700 1 -- 172.21.15.7:0/2692549638 learned_addr learned my addr 172.21.15.7:0/2692549638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca37fe700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 -- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 msgr2=0x7efca40a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 -- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 msgr2=0x7efca40aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 0x7efca40aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 -- 172.21.15.7:0/2692549638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c009a20 con 0x7efca40ca470 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca8e01700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 0x7efca41693d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e70 tx=0x7efc9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00caa0 con 0x7efca40ca470 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00f040 con 0x7efca40ca470 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc94009420 con 0x7efca40ca470 2023-10-06T11:23:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00ce90 con 0x7efca40ca470 2023-10-06T11:23:43.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.959+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40c27c0 con 0x7efca40ca470 2023-10-06T11:23:43.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40c29e0 con 0x7efca40ca470 2023-10-06T11:23:43.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7efc9c003020 con 0x7efca40ca470 2023-10-06T11:23:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efc9bfff700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 0x7efc84062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7efc9c07de00 con 0x7efca40ca470 2023-10-06T11:23:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efca3fff700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 0x7efc84062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.963+0000 7efca3fff700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 0x7efc84062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c010 tx=0x7efc94017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:43.967+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efc9c04e080 con 0x7efca40ca470 2023-10-06T11:23:44.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.123+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7efca40c2640 con 0x7efc84060670 2023-10-06T11:23:44.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.123+0000 7efc9bfff700 1 -- 172.21.15.7:0/2692549638 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc9400b910 con 0x7efc84060670 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 msgr2=0x7efc84062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 0x7efc84062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c010 tx=0x7efc94017000 comp rx=0 tx=0).stop 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 msgr2=0x7efca41693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 0x7efca41693d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e70 tx=0x7efc9c00b040 comp rx=0 tx=0).stop 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 shutdown_connections 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7efc84060670 0x7efc84062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca40ca470 0x7efca41693d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7efca40afb00 0x7efca40a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 --2- 172.21.15.7:0/2692549638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efca40ade50 0x7efca40aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.127+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 >> 172.21.15.7:0/2692549638 conn(0x7efca406bd70 msgr2=0x7efca40ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.131+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 shutdown_connections 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.131+0000 7efca9602700 1 -- 172.21.15.7:0/2692549638 wait complete. 2023-10-06T11:23:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T11:23:44.148 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T11:23:44.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:23:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 Processor -- start 2023-10-06T11:23:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- start start 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2806df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7feb28159c10 0x7feb2806a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb280cc140 0x7feb28154d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb280c7c80 con 0x7feb280cc140 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb280c6dd0 con 0x7feb28159c10 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280c7070 con 0x7feb2806db70 2023-10-06T11:23:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb280cc140 0x7feb28154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb280cc140 0x7feb28154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59562/0 (socket says 172.21.15.7:59562) 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 -- 172.21.15.7:0/862615560 learned_addr learned my addr 172.21.15.7:0/862615560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb1f7fe700 1 --2- 172.21.15.7:0/862615560 >> v2:172.21.15.50:3300/0 conn(0x7feb28159c10 0x7feb2806a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb27fff700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2806df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 -- 172.21.15.7:0/862615560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 msgr2=0x7feb2806df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2806df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 -- 172.21.15.7:0/862615560 >> v2:172.21.15.50:3300/0 conn(0x7feb28159c10 msgr2=0x7feb2806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 --2- 172.21.15.7:0/862615560 >> v2:172.21.15.50:3300/0 conn(0x7feb28159c10 0x7feb2806a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 -- 172.21.15.7:0/862615560 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280c7350 con 0x7feb280cc140 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb27fff700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2806df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2c818700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb28154d10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb2806d630 tx=0x7feb20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d507f5eba7280394 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb277fe700 1 -- 172.21.15.7:0/862615560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200092e0 con 0x7feb280cc140 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb277fe700 1 -- 172.21.15.7:0/862615560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2000ba20 con 0x7feb280cc140 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb277fe700 1 -- 172.21.15.7:0/862615560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2000bde0 con 0x7feb280cc140 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- 172.21.15.7:0/862615560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 msgr2=0x7feb28154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb28154d10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb2806d630 tx=0x7feb20009840 comp rx=0 tx=0).stop 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- 172.21.15.7:0/862615560 shutdown_connections 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb28154d10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- 172.21.15.7:0/862615560 >> v2:172.21.15.50:3300/0 conn(0x7feb28159c10 0x7feb2806a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 --2- 172.21.15.7:0/862615560 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2806df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- 172.21.15.7:0/862615560 >> 172.21.15.7:0/862615560 conn(0x7feb28068900 msgr2=0x7feb280cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- 172.21.15.7:0/862615560 shutdown_connections 2023-10-06T11:23:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.259+0000 7feb2e27b700 1 -- 172.21.15.7:0/862615560 wait complete. 2023-10-06T11:23:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.263+0000 7feb2e27b700 1 Processor -- start 2023-10-06T11:23:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.263+0000 7feb2e27b700 1 -- start start 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb280c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb28159c10 0x7feb280b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280b01d0 con 0x7feb280cc140 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280af5c0 con 0x7feb28159c10 2023-10-06T11:23:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280af8a0 con 0x7feb2806db70 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb280c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb280c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59572/0 (socket says 172.21.15.7:59572) 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 -- 172.21.15.7:0/1263140860 learned_addr learned my addr 172.21.15.7:0/1263140860 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb27fff700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2c818700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb28159c10 0x7feb280b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 -- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 msgr2=0x7feb2814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7feb2806db70 0x7feb2814f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 -- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb28159c10 msgr2=0x7feb280b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb28159c10 0x7feb280b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009ae0 con 0x7feb280cc140 2023-10-06T11:23:44.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2c818700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb28159c10 0x7feb280b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:23:44.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb1f7fe700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb280c4ae0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e930 tx=0x7feb100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10009510 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1000c850 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb280afb80 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000cc10 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb280af190 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.267+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb2814d660 con 0x7feb280cc140 2023-10-06T11:23:44.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7feb10013470 con 0x7feb280cc140 2023-10-06T11:23:44.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7feb08060670 0x7feb08062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7feb1007d0b0 con 0x7feb280cc140 2023-10-06T11:23:44.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7feb08067480 0x7feb080698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb08069f80 con 0x7feb08067480 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb27fff700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7feb08060670 0x7feb08062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb2c818700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7feb08067480 0x7feb080698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb1004d3c0 con 0x7feb280cc140 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb27fff700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7feb08060670 0x7feb08062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb18006f00 tx=0x7feb18008040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.271+0000 7feb2c818700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7feb08067480 0x7feb080698b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.275+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== osd.7 v2:172.21.15.50:6817/33667 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7feb08069f80 con 0x7feb08067480 2023-10-06T11:23:44.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.291+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 --> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb2804be00 con 0x7feb08067480 2023-10-06T11:23:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.291+0000 7feb24ff9700 1 -- 172.21.15.7:0/1263140860 <== osd.7 v2:172.21.15.50:6817/33667 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feb2804be00 con 0x7feb08067480 2023-10-06T11:23:44.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-06T11:23:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:23:44.297 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.291+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7feb08067480 msgr2=0x7feb080698b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.291+0000 7feb2e27b700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6817/33667,v1:172.21.15.50:6820/33667] conn(0x7feb08067480 0x7feb080698b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7feb08060670 msgr2=0x7feb08062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7feb08060670 0x7feb08062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb18006f00 tx=0x7feb18008040 comp rx=0 tx=0).stop 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 msgr2=0x7feb280c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 --2- 172.21.15.7:0/1263140860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb280cc140 0x7feb280c4ae0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e930 tx=0x7feb100091a0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2c818700 1 -- 172.21.15.7:0/1263140860 reap_dead start 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 shutdown_connections 2023-10-06T11:23:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 >> 172.21.15.7:0/1263140860 conn(0x7feb28068900 msgr2=0x7feb2806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 shutdown_connections 2023-10-06T11:23:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.295+0000 7feb2e27b700 1 -- 172.21.15.7:0/1263140860 wait complete. 2023-10-06T11:23:44.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T11:23:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 Processor -- start 2023-10-06T11:23:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 -- start start 2023-10-06T11:23:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4910159080 0x7f4910157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49100b0730 con 0x7f49100afcb0 2023-10-06T11:23:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4910156a60 con 0x7f4910069da0 2023-10-06T11:23:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4917d51700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4910156d40 con 0x7f4910159080 2023-10-06T11:23:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4915aed700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4910159080 0x7f4910157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f49152ec700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f49100afcb0 0x7f4910156180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54474/0 (socket says 172.21.15.7:54474) 2023-10-06T11:23:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f49152ec700 1 -- 172.21.15.7:0/2168556799 learned_addr learned my addr 172.21.15.7:0/2168556799 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:23:44.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4914aeb700 1 -- 172.21.15.7:0/2168556799 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91922758 0 0) 0x7f49100b0730 con 0x7f49100afcb0 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.415+0000 7f4914aeb700 1 -- 172.21.15.7:0/2168556799 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48fc003570 con 0x7f49100afcb0 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4915aed700 1 -- 172.21.15.7:0/2168556799 >> v1:172.21.15.50:6789/0 conn(0x7f4910069da0 legacy=0x7f491006a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4915aed700 1 -- 172.21.15.7:0/2168556799 >> v1:172.21.15.7:6789/0 conn(0x7f49100afcb0 legacy=0x7f4910156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4915aed700 1 -- 172.21.15.7:0/2168556799 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910157020 con 0x7f4910159080 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4915aed700 1 --2- 172.21.15.7:0/2168556799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f4910157390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d0d0 tx=0x7f490c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc3ce3f63257232 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4914aeb700 1 -- 172.21.15.7:0/2168556799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c010070 con 0x7f4910159080 2023-10-06T11:23:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4914aeb700 1 -- 172.21.15.7:0/2168556799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c00b2e0 con 0x7f4910159080 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4914aeb700 1 -- 172.21.15.7:0/2168556799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00dd80 con 0x7f4910159080 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- 172.21.15.7:0/2168556799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 msgr2=0x7f4910157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 --2- 172.21.15.7:0/2168556799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f4910157390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d0d0 tx=0x7f490c00b990 comp rx=0 tx=0).stop 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- 172.21.15.7:0/2168556799 shutdown_connections 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 --2- 172.21.15.7:0/2168556799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f4910157390 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- 172.21.15.7:0/2168556799 >> 172.21.15.7:0/2168556799 conn(0x7f4910067e30 msgr2=0x7f4910068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- 172.21.15.7:0/2168556799 shutdown_connections 2023-10-06T11:23:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- 172.21.15.7:0/2168556799 wait complete. 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 Processor -- start 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- start start 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4910069da0 0x7f4910169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f49100afcb0 0x7f4910169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491016a400 con 0x7f4910159080 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491014ef90 con 0x7f4910069da0 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.419+0000 7f4917d51700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491016a630 con 0x7f49100afcb0 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4915aed700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4910069da0 0x7f4910169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59590/0 (socket says 172.21.15.7:59590) 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 -- 172.21.15.7:0/1159230891 learned_addr learned my addr 172.21.15.7:0/1159230891 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49152ec700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f49100afcb0 0x7f4910169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 -- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f49100afcb0 msgr2=0x7f4910169ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f49100afcb0 0x7f4910169ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 -- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4910069da0 msgr2=0x7f4910169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4910069da0 0x7f4910169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 -- 172.21.15.7:0/1159230891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c00bc30 con 0x7f4910159080 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f49162ee700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f490800c560 tx=0x7f4908007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908011590 con 0x7f4910159080 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908011710 con 0x7f4910159080 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491016a910 con 0x7f4910159080 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908011ad0 con 0x7f4910159080 2023-10-06T11:23:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491016aed0 con 0x7f4910159080 2023-10-06T11:23:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4910070f20 con 0x7f4910159080 2023-10-06T11:23:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.423+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f490801b3b0 con 0x7f4910159080 2023-10-06T11:23:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.427+0000 7f4905ffb700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 0x7f48f4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.427+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f490807bf40 con 0x7f4910159080 2023-10-06T11:23:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.427+0000 7f4915aed700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 0x7f48f4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.427+0000 7f4915aed700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 0x7f48f4062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f490c00bc00 tx=0x7f490c01a000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.431+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f490804c1b0 con 0x7f4910159080 2023-10-06T11:23:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.595+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f49100cd070 con 0x7f48f4060670 2023-10-06T11:23:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.595+0000 7f4905ffb700 1 -- 172.21.15.7:0/1159230891 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49100cd070 con 0x7f48f4060670 2023-10-06T11:23:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 msgr2=0x7f48f4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 0x7f48f4062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f490c00bc00 tx=0x7f490c01a000 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 msgr2=0x7f491014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f490800c560 tx=0x7f4908007ef0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 shutdown_connections 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f48f4060670 0x7f48f4062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4910159080 0x7f491014ea50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f49100afcb0 0x7f4910169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 --2- 172.21.15.7:0/1159230891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4910069da0 0x7f4910169840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 >> 172.21.15.7:0/1159230891 conn(0x7f4910067e30 msgr2=0x7f491014fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 shutdown_connections 2023-10-06T11:23:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.599+0000 7f4917d51700 1 -- 172.21.15.7:0/1159230891 wait complete. 2023-10-06T11:23:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T11:23:44.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:23:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.707+0000 7fb0c7fff700 1 Processor -- start 2023-10-06T11:23:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.707+0000 7fb0c7fff700 1 -- start start 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0c8158d90 0x7fb0c8066840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb0c8068540 0x7fb0c80ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80c78b0 con 0x7fb0c8159160 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80c7280 con 0x7fb0c8158d90 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c8068a40 con 0x7fb0c8068540 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0c8158d90 0x7fb0c8066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0c8158d90 0x7fb0c8066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59592/0 (socket says 172.21.15.7:59592) 2023-10-06T11:23:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 -- 172.21.15.7:0/2989233785 learned_addr learned my addr 172.21.15.7:0/2989233785 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c77fe700 1 --2- 172.21.15.7:0/2989233785 >> v2:172.21.15.50:3300/0 conn(0x7fb0c8068540 0x7fb0c80ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 -- 172.21.15.7:0/2989233785 >> v2:172.21.15.50:3300/0 conn(0x7fb0c8068540 msgr2=0x7fb0c80ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 --2- 172.21.15.7:0/2989233785 >> v2:172.21.15.50:3300/0 conn(0x7fb0c8068540 0x7fb0c80ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 -- 172.21.15.7:0/2989233785 >> v1:172.21.15.50:6789/0 conn(0x7fb0c8159160 legacy=0x7fb0c80c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c5ffb700 1 -- 172.21.15.7:0/2989233785 <== mon.1 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850538518 0 0) 0x7fb0c80c78b0 con 0x7fb0c8159160 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 -- 172.21.15.7:0/2989233785 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80c7540 con 0x7fb0c8158d90 2023-10-06T11:23:44.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c6ffd700 1 --2- 172.21.15.7:0/2989233785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c8158d90 0x7fb0c8066840 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80c82b0 tx=0x7fb0b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e7c7bf1e392c73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c5ffb700 1 -- 172.21.15.7:0/2989233785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0009e70 con 0x7fb0c8158d90 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c5ffb700 1 -- 172.21.15.7:0/2989233785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b000b860 con 0x7fb0c8158d90 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c5ffb700 1 -- 172.21.15.7:0/2989233785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000bc00 con 0x7fb0c8158d90 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- 172.21.15.7:0/2989233785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c8158d90 msgr2=0x7fb0c8066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/2989233785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c8158d90 0x7fb0c8066840 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80c82b0 tx=0x7fb0b0009250 comp rx=0 tx=0).stop 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- 172.21.15.7:0/2989233785 shutdown_connections 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/2989233785 >> v2:172.21.15.50:3300/0 conn(0x7fb0c8068540 0x7fb0c80ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/2989233785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c8158d90 0x7fb0c8066840 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- 172.21.15.7:0/2989233785 >> 172.21.15.7:0/2989233785 conn(0x7fb0c806a990 msgr2=0x7fb0c8067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- 172.21.15.7:0/2989233785 shutdown_connections 2023-10-06T11:23:44.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.711+0000 7fb0c7fff700 1 -- 172.21.15.7:0/2989233785 wait complete. 2023-10-06T11:23:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 Processor -- start 2023-10-06T11:23:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 -- start start 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c8068540 0x7fb0c80ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb0c8158d90 0x7fb0c814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c816a200 con 0x7fb0c814edd0 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c81695f0 con 0x7fb0c8068540 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c81698d0 con 0x7fb0c8158d90 2023-10-06T11:23:44.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59602/0 (socket says 172.21.15.7:59602) 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c8068540 0x7fb0c80ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 -- 172.21.15.7:0/895433637 learned_addr learned my addr 172.21.15.7:0/895433637 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c67fc700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb0c8158d90 0x7fb0c814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 -- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb0c8158d90 msgr2=0x7fb0c814e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb0c8158d90 0x7fb0c814e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 -- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c8068540 msgr2=0x7fb0c80ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c8068540 0x7fb0c80ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 -- 172.21.15.7:0/895433637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00094f0 con 0x7fb0c814edd0 2023-10-06T11:23:44.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c77fe700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ef0 tx=0x7fb0c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000cb70 con 0x7fb0c814edd0 2023-10-06T11:23:44.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000f040 con 0x7fb0c814edd0 2023-10-06T11:23:44.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c8169bb0 con 0x7fb0c814edd0 2023-10-06T11:23:44.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.715+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0014480 con 0x7fb0c814edd0 2023-10-06T11:23:44.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.719+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c80ab690 con 0x7fb0c814edd0 2023-10-06T11:23:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.719+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb0c0014830 con 0x7fb0c814edd0 2023-10-06T11:23:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.719+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c816a3b0 con 0x7fb0c814edd0 2023-10-06T11:23:44.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.719+0000 7fb0beffd700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 0x7fb0ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:23:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.719+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fb0c007d8b0 con 0x7fb0c814edd0 2023-10-06T11:23:44.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.723+0000 7fb0c6ffd700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 0x7fb0ac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:23:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.723+0000 7fb0c6ffd700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 0x7fb0ac062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80c82b0 tx=0x7fb0b0017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:23:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.731+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb0c816a3b0 con 0x7fb0c814edd0 2023-10-06T11:23:44.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.887+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0c806b8a0 con 0x7fb0ac060670 2023-10-06T11:23:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.891+0000 7fb0beffd700 1 -- 172.21.15.7:0/895433637 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70896 (secure 0 0 0) 0x7fb0c806b8a0 con 0x7fb0ac060670 2023-10-06T11:23:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 msgr2=0x7fb0ac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 0x7fb0ac062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80c82b0 tx=0x7fb0b0017000 comp rx=0 tx=0).stop 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 msgr2=0x7fb0c80aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ef0 tx=0x7fb0c000b040 comp rx=0 tx=0).stop 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 shutdown_connections 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fb0ac060670 0x7fb0ac062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c814edd0 0x7fb0c80aef00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fb0c8158d90 0x7fb0c814e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 --2- 172.21.15.7:0/895433637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c8068540 0x7fb0c80ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 >> 172.21.15.7:0/895433637 conn(0x7fb0c806a990 msgr2=0x7fb0c80c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 shutdown_connections 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:23:44.895+0000 7fb0c7fff700 1 -- 172.21.15.7:0/895433637 wait complete. 2023-10-06T11:23:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-10-06T11:23:44.018227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2416296,"kb_used_data":5224,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752558424,"statfs":{"total":773094113280,"available":770619826176,"internally_reserved":0,"allocated":5349376,"data_stored":3084048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.409245"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.983528+0000","last_change":"2023-10-06T11:21:52.281804+0000","last_active":"2023-10-06T11:23:34.983528+0000","last_peered":"2023-10-06T11:23:34.983528+0000","last_clean":"2023-10-06T11:23:34.983528+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:34.983528+0000","last_undegraded":"2023-10-06T11:23:34.983528+0000","last_fullsized":"2023-10-06T11:23:34.983528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.984358+0000","last_change":"2023-10-06T11:21:52.285694+0000","last_active":"2023-10-06T11:23:34.984358+0000","last_peered":"2023-10-06T11:23:34.984358+0000","last_clean":"2023-10-06T11:23:34.984358+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:34.984358+0000","last_undegraded":"2023-10-06T11:23:34.984358+0000","last_fullsized":"2023-10-06T11:23:34.984358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.986495+0000","last_change":"2023-10-06T11:21:52.286031+0000","last_active":"2023-10-06T11:23:34.986495+0000","last_peered":"2023-10-06T11:23:34.986495+0000","last_clean":"2023-10-06T11:23:34.986495+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:34.986495+0000","last_undegraded":"2023-10-06T11:23:34.986495+0000","last_fullsized":"2023-10-06T11:23:34.986495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:36.628627+0000","last_change":"2023-10-06T11:21:52.165497+0000","last_active":"2023-10-06T11:23:36.628627+0000","last_peered":"2023-10-06T11:23:36.628627+0000","last_clean":"2023-10-06T11:23:36.628627+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:36.628627+0000","last_undegraded":"2023-10-06T11:23:36.628627+0000","last_fullsized":"2023-10-06T11:23:36.628627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'139","reported_seq":262,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.983358+0000","last_change":"2023-10-06T11:21:53.180861+0000","last_active":"2023-10-06T11:23:34.983358+0000","last_peered":"2023-10-06T11:23:34.983358+0000","last_clean":"2023-10-06T11:23:34.983358+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:34.983358+0000","last_undegraded":"2023-10-06T11:23:34.983358+0000","last_fullsized":"2023-10-06T11:23:34.983358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:52.147747+0000","last_clean_scrub_stamp":"2023-10-06T11:21:52.147747+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:48.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:35.618156+0000","last_change":"2023-10-06T11:21:52.165618+0000","last_active":"2023-10-06T11:23:35.618156+0000","last_peered":"2023-10-06T11:23:35.618156+0000","last_clean":"2023-10-06T11:23:35.618156+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:35.618156+0000","last_undegraded":"2023-10-06T11:23:35.618156+0000","last_fullsized":"2023-10-06T11:23:35.618156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.613662+0000","last_change":"2023-10-06T11:21:52.163246+0000","last_active":"2023-10-06T11:23:34.613662+0000","last_peered":"2023-10-06T11:23:34.613662+0000","last_clean":"2023-10-06T11:23:34.613662+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:34.613662+0000","last_undegraded":"2023-10-06T11:23:34.613662+0000","last_fullsized":"2023-10-06T11:23:34.613662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:52.423648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.669750+0000","last_change":"2023-10-06T11:21:52.281805+0000","last_active":"2023-10-06T11:23:34.669750+0000","last_peered":"2023-10-06T11:23:34.669750+0000","last_clean":"2023-10-06T11:23:34.669750+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:34.669750+0000","last_undegraded":"2023-10-06T11:23:34.669750+0000","last_fullsized":"2023-10-06T11:23:34.669750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.327595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T11:23:34.670321+0000","last_change":"2023-10-06T11:21:52.168789+0000","last_active":"2023-10-06T11:23:34.670321+0000","last_peered":"2023-10-06T11:23:34.670321+0000","last_clean":"2023-10-06T11:23:34.670321+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:34.670321+0000","last_undegraded":"2023-10-06T11:23:34.670321+0000","last_fullsized":"2023-10-06T11:23:34.670321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:21:51.144326+0000","last_clean_scrub_stamp":"2023-10-06T11:21:51.144326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:22.166752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":828282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.024},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.1220000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.873},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.234}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.1100000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.629},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.0589999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.1439999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.214},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.177}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.173}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.747}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.86499999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.52}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.01},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.585},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.2090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.077}]},{"osd":1,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.147}]},{"osd":3,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":1.042}]},{"osd":5,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.206}]},{"osd":7,"last update":"Fri Oct 6 11:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.099},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.173}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.73899999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":1.054}]},{"osd":4,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1859999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.25}]},{"osd":7,"last update":"Fri Oct 6 11:22:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.2270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.68200000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.623},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.1439999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.202},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.238}]},{"osd":6,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.415},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.444}]},{"osd":7,"last update":"Fri Oct 6 11:22:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.333}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":828282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.171}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.069}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.0129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.032},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.499},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.92700000000000005}]},{"osd":4,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.111}]},{"osd":5,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.96199999999999997}]},{"osd":6,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0940000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.90900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:23:44.917 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.917 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.917 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T11:21:52.147747+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T11:21:51.144326+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=21, tm_sec=51, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=23, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:23:44.918 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T11:24:04.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:24:05.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.030+0000 7f623b59e700 1 Processor -- start 2023-10-06T11:24:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.030+0000 7f623b59e700 1 -- start start 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f623c0cab80 0x7f623c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c0ca0a0 con 0x7f623c0afb00 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c157230 con 0x7f623c0ad740 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0ae2d0 con 0x7f623c0cab80 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f623c0afb00 0x7f623c0ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51654/0 (socket says 172.21.15.7:51654) 2023-10-06T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623a59c700 1 -- 172.21.15.7:0/1605296118 learned_addr learned my addr 172.21.15.7:0/1605296118 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-06T11:24:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6231d9b700 1 --2- 172.21.15.7:0/1605296118 >> v2:172.21.15.50:3300/0 conn(0x7f623c0cab80 0x7f623c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6239d9b700 1 -- 172.21.15.7:0/1605296118 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320534975 0 0) 0x7f623c0ca0a0 con 0x7f623c0afb00 2023-10-06T11:24:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6239d9b700 1 -- 172.21.15.7:0/1605296118 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f621c003570 con 0x7f623c0afb00 2023-10-06T11:24:05.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6231d9b700 1 -- 172.21.15.7:0/1605296118 >> v1:172.21.15.50:6789/0 conn(0x7f623c0ad740 legacy=0x7f623c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6231d9b700 1 -- 172.21.15.7:0/1605296118 >> v1:172.21.15.7:6789/0 conn(0x7f623c0afb00 legacy=0x7f623c0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6231d9b700 1 -- 172.21.15.7:0/1605296118 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c157510 con 0x7f623c0cab80 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6231d9b700 1 --2- 172.21.15.7:0/1605296118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0cab80 0x7f623c0afed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098e0 tx=0x7f622c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ae26e1403e033f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6239d9b700 1 -- 172.21.15.7:0/1605296118 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00b640 con 0x7f623c0cab80 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6239d9b700 1 -- 172.21.15.7:0/1605296118 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c00b7a0 con 0x7f623c0cab80 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f6239d9b700 1 -- 172.21.15.7:0/1605296118 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00bb20 con 0x7f623c0cab80 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- 172.21.15.7:0/1605296118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0cab80 msgr2=0x7f623c0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 --2- 172.21.15.7:0/1605296118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0cab80 0x7f623c0afed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098e0 tx=0x7f622c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- 172.21.15.7:0/1605296118 shutdown_connections 2023-10-06T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 --2- 172.21.15.7:0/1605296118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0cab80 0x7f623c0afed0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- 172.21.15.7:0/1605296118 >> 172.21.15.7:0/1605296118 conn(0x7f623c06bd70 msgr2=0x7f623c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- 172.21.15.7:0/1605296118 shutdown_connections 2023-10-06T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.034+0000 7f623b59e700 1 -- 172.21.15.7:0/1605296118 wait complete. 2023-10-06T11:24:05.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 Processor -- start 2023-10-06T11:24:05.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 -- start start 2023-10-06T11:24:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 0x7f623c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f623c0cab80 0x7f623c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c169a90 con 0x7f623c0ad740 2023-10-06T11:24:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c068f40 con 0x7f623c0afb00 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623b59e700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c069220 con 0x7f623c0cab80 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 0x7f623c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f6231d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f623ad9d700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f623c0cab80 0x7f623c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f6231d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.7:53344/0 (socket says 172.21.15.7:53344) 2023-10-06T11:24:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.038+0000 7f6231d9b700 1 -- 172.21.15.7:0/3043065736 learned_addr learned my addr 172.21.15.7:0/3043065736 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 -- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f623c0cab80 msgr2=0x7f623c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f623c0cab80 0x7f623c0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 -- 172.21.15.7:0/3043065736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 msgr2=0x7f623c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 0x7f623c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 -- 172.21.15.7:0/3043065736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009420 con 0x7f623c0afb00 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f623a59c700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 0x7f623c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6231d9b700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f623c0ca230 tx=0x7f622c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00bb20 con 0x7f623c0afb00 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c000d10 con 0x7f623c0afb00 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c1565b0 con 0x7f623c0afb00 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c011630 con 0x7f623c0afb00 2023-10-06T11:24:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c0c7990 con 0x7f623c0afb00 2023-10-06T11:24:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.042+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c06ec60 con 0x7f623c0afb00 2023-10-06T11:24:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.046+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f622c011a60 con 0x7f623c0afb00 2023-10-06T11:24:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.046+0000 7f6232ffd700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 0x7f6220062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.046+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f622c07eb70 con 0x7f623c0afb00 2023-10-06T11:24:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.046+0000 7f623a59c700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 0x7f6220062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.046+0000 7f623a59c700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 0x7f6220062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6224007fd0 tx=0x7f6224009040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.054+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f622c04ef00 con 0x7f623c0afb00 2023-10-06T11:24:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.210+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 --> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f623c0b3330 con 0x7f6220060670 2023-10-06T11:24:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.218+0000 7f6232ffd700 1 -- 172.21.15.7:0/3043065736 <== mgr.5458 v2:172.21.15.50:6832/33317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70891 (secure 0 0 0) 0x7f623c0b3330 con 0x7f6220060670 2023-10-06T11:24:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 msgr2=0x7f6220062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 0x7f6220062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6224007fd0 tx=0x7f6224009040 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 msgr2=0x7f623c0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f623c0ca230 tx=0x7f622c017000 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 shutdown_connections 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7f6220060670 0x7f6220062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7f623c0cab80 0x7f623c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f623c0afb00 0x7f623c0ce250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 --2- 172.21.15.7:0/3043065736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0ad740 0x7f623c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 >> 172.21.15.7:0/3043065736 conn(0x7f623c06bd70 msgr2=0x7f623c159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 shutdown_connections 2023-10-06T11:24:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.222+0000 7f623b59e700 1 -- 172.21.15.7:0/3043065736 wait complete. 2023-10-06T11:24:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-06T11:24:05.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:24:04.024155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2416296,"kb_used_data":5224,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752558424,"statfs":{"total":773094113280,"available":770619826176,"internally_reserved":0,"allocated":5349376,"data_stored":3084048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003583"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:47.675323+0000","last_change":"2023-10-06T11:23:47.675178+0000","last_active":"2023-10-06T11:23:47.675323+0000","last_peered":"2023-10-06T11:23:47.675323+0000","last_clean":"2023-10-06T11:23:47.675323+0000","last_became_active":"2023-10-06T11:21:52.281365+0000","last_became_peered":"2023-10-06T11:21:52.281365+0000","last_unstale":"2023-10-06T11:23:47.675323+0000","last_undegraded":"2023-10-06T11:23:47.675323+0000","last_fullsized":"2023-10-06T11:23:47.675323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:47.674912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:47.674912+0000","last_clean_scrub_stamp":"2023-10-06T11:23:47.674912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:48.995735+0000","scrub_duration":0.0049008990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:43.681932+0000","last_change":"2023-10-06T11:23:43.681833+0000","last_active":"2023-10-06T11:23:43.681932+0000","last_peered":"2023-10-06T11:23:43.681932+0000","last_clean":"2023-10-06T11:23:43.681932+0000","last_became_active":"2023-10-06T11:21:52.285290+0000","last_became_peered":"2023-10-06T11:21:52.285290+0000","last_unstale":"2023-10-06T11:23:43.681932+0000","last_undegraded":"2023-10-06T11:23:43.681932+0000","last_fullsized":"2023-10-06T11:23:43.681932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:43.681587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:43.681587+0000","last_clean_scrub_stamp":"2023-10-06T11:23:43.681587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:01.006186+0000","scrub_duration":0.0042033319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":46,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:45.723998+0000","last_change":"2023-10-06T11:23:45.723921+0000","last_active":"2023-10-06T11:23:45.723998+0000","last_peered":"2023-10-06T11:23:45.723998+0000","last_clean":"2023-10-06T11:23:45.723998+0000","last_became_active":"2023-10-06T11:21:52.285523+0000","last_became_peered":"2023-10-06T11:21:52.285523+0000","last_unstale":"2023-10-06T11:23:45.723998+0000","last_undegraded":"2023-10-06T11:23:45.723998+0000","last_fullsized":"2023-10-06T11:23:45.723998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:45.723648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:45.723648+0000","last_clean_scrub_stamp":"2023-10-06T11:23:45.723648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:19.922001+0000","scrub_duration":0.0048719779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":44,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:44.612243+0000","last_change":"2023-10-06T11:23:44.612216+0000","last_active":"2023-10-06T11:23:44.612243+0000","last_peered":"2023-10-06T11:23:44.612243+0000","last_clean":"2023-10-06T11:23:44.612243+0000","last_became_active":"2023-10-06T11:21:52.165070+0000","last_became_peered":"2023-10-06T11:21:52.165070+0000","last_unstale":"2023-10-06T11:23:44.612243+0000","last_undegraded":"2023-10-06T11:23:44.612243+0000","last_fullsized":"2023-10-06T11:23:44.612243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:44.612116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:44.612116+0000","last_clean_scrub_stamp":"2023-10-06T11:23:44.612116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:05.715542+0000","scrub_duration":0.0036071480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'140","reported_seq":274,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:48.680972+0000","last_change":"2023-10-06T11:23:48.680902+0000","last_active":"2023-10-06T11:23:48.680972+0000","last_peered":"2023-10-06T11:23:48.680972+0000","last_clean":"2023-10-06T11:23:48.680972+0000","last_became_active":"2023-10-06T11:21:53.180403+0000","last_became_peered":"2023-10-06T11:21:53.180403+0000","last_unstale":"2023-10-06T11:23:48.680972+0000","last_undegraded":"2023-10-06T11:23:48.680972+0000","last_fullsized":"2023-10-06T11:23:48.680972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'140","last_scrub_stamp":"2023-10-06T11:23:48.680666+0000","last_deep_scrub":"44'140","last_deep_scrub_stamp":"2023-10-06T11:23:48.680666+0000","last_clean_scrub_stamp":"2023-10-06T11:23:48.680666+0000","objects_scrubbed":2,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:21:11.104935+0000","scrub_duration":0.010075749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":47,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:46.481082+0000","last_change":"2023-10-06T11:23:46.481009+0000","last_active":"2023-10-06T11:23:46.481082+0000","last_peered":"2023-10-06T11:23:46.481082+0000","last_clean":"2023-10-06T11:23:46.481082+0000","last_became_active":"2023-10-06T11:21:52.165219+0000","last_became_peered":"2023-10-06T11:21:52.165219+0000","last_unstale":"2023-10-06T11:23:46.481082+0000","last_undegraded":"2023-10-06T11:23:46.481082+0000","last_fullsized":"2023-10-06T11:23:46.481082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:46.480685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:46.480685+0000","last_clean_scrub_stamp":"2023-10-06T11:23:46.480685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:42.823547+0000","scrub_duration":0.0047579400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":46,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:45.624821+0000","last_change":"2023-10-06T11:23:45.624756+0000","last_active":"2023-10-06T11:23:45.624821+0000","last_peered":"2023-10-06T11:23:45.624821+0000","last_clean":"2023-10-06T11:23:45.624821+0000","last_became_active":"2023-10-06T11:21:52.162912+0000","last_became_peered":"2023-10-06T11:21:52.162912+0000","last_unstale":"2023-10-06T11:23:45.624821+0000","last_undegraded":"2023-10-06T11:23:45.624821+0000","last_fullsized":"2023-10-06T11:23:45.624821+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:45.624521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:45.624521+0000","last_clean_scrub_stamp":"2023-10-06T11:23:45.624521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:16.781573+0000","scrub_duration":0.0045508450000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":44,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:46.003446+0000","last_change":"2023-10-06T11:23:46.003374+0000","last_active":"2023-10-06T11:23:46.003446+0000","last_peered":"2023-10-06T11:23:46.003446+0000","last_clean":"2023-10-06T11:23:46.003446+0000","last_became_active":"2023-10-06T11:21:52.281334+0000","last_became_peered":"2023-10-06T11:21:52.281334+0000","last_unstale":"2023-10-06T11:23:46.003446+0000","last_undegraded":"2023-10-06T11:23:46.003446+0000","last_fullsized":"2023-10-06T11:23:46.003446+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:46.003132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:46.003132+0000","last_clean_scrub_stamp":"2023-10-06T11:23:46.003132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:33.152469+0000","scrub_duration":0.0041757599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T11:23:44.975079+0000","last_change":"2023-10-06T11:23:44.974954+0000","last_active":"2023-10-06T11:23:44.975079+0000","last_peered":"2023-10-06T11:23:44.975079+0000","last_clean":"2023-10-06T11:23:44.975079+0000","last_became_active":"2023-10-06T11:21:52.168469+0000","last_became_peered":"2023-10-06T11:21:52.168469+0000","last_unstale":"2023-10-06T11:23:44.975079+0000","last_undegraded":"2023-10-06T11:23:44.975079+0000","last_fullsized":"2023-10-06T11:23:44.975079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:23:44.974705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:23:44.974705+0000","last_clean_scrub_stamp":"2023-10-06T11:23:44.974705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:18:06.078130+0000","scrub_duration":0.004156491,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":828282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.95199999999999996,"15min":0.96299999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3340000000000001,"5min":1.591,"15min":1.591},"last":1.018},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.89300000000000002,"15min":0.88500000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.93700000000000006}]},{"osd":1,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.272,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.381,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.494}]},{"osd":2,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.024,"15min":1.026},"min":{"1min":0.72499999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3640000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0600000000000001,"15min":1.0700000000000001},"min":{"1min":0.68899999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4099999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.536}]},{"osd":3,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.95199999999999996,"15min":0.94199999999999995},"min":{"1min":0.73999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.157},{"interface":"front","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1319999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3819999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.026,"15min":1.036},"min":{"1min":0.73999999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.25,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.111,"15min":1.119},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3440000000000001,"5min":1.399,"15min":1.399},"last":1.355}]},{"osd":5,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2509999999999999,"15min":1.2689999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4399999999999999,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.421},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2789999999999999,"15min":1.2809999999999999},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":1.78,"5min":1.792,"15min":1.792},"last":1.6859999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2470000000000001,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.5209999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2629999999999999,"15min":1.268},"min":{"1min":1.0209999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.4550000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56299999999999994,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.115,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.96499999999999997,"15min":0.97099999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.196,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.749},{"interface":"front","average":{"1min":1.006,"5min":1.0649999999999999,"15min":1.075},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2170000000000001,"5min":1.694,"15min":1.694},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.024,"15min":1.036},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.105,"15min":1.111},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.032,"15min":1.0289999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4490000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.238},{"interface":"front","average":{"1min":1.139,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.087}]},{"osd":4,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.077},"min":{"1min":0.64700000000000002,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3899999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6850000000000001,"5min":1.748,"15min":1.748},"last":1.1160000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1970000000000001,"15min":1.2030000000000001},"min":{"1min":0.75,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.29,"15min":1.294},"min":{"1min":0.96199999999999997,"5min":0.751,"15min":0.751},"max":{"1min":1.5569999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.29}]},{"osd":7,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.306,"15min":1.3089999999999999},"min":{"1min":1.022,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6890000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.357},{"interface":"front","average":{"1min":1.319,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":1.0549999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.204}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.86599999999999999,"15min":0.88200000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85299999999999998,"15min":0.85099999999999998},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.0860000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.89900000000000002,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.96299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1919999999999999,"5min":1.341,"15min":1.341},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95899999999999996,"15min":0.96299999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.357,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.93799999999999994,"15min":0.94199999999999995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.226,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.2889999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94599999999999995,"15min":0.94499999999999995},"min":{"1min":0.69099999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0549999999999999,"15min":1.0669999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.3620000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.2150000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.94499999999999995,"15min":0.95099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.145,"15min":1.1459999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.177}]},{"osd":6,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.115,"15min":1.123},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4359999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.127},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.119,"15min":1.1240000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.4219999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1040000000000001,"15min":1.113},"min":{"1min":0.71199999999999997,"5min":0.62,"15min":0.62},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.2010000000000001,"15min":1.2050000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.409,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.349}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.91000000000000003,"15min":0.90200000000000002},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.226,"5min":1.575,"15min":1.575},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.99399999999999999,"15min":0.98899999999999999},"min":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.494,"5min":1.903,"15min":1.903},"last":0.872},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.68100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4410000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.02,"15min":1.03},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.2629999999999999,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.129,"15min":1.129},"min":{"1min":0.751,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.613,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98199999999999998,"15min":0.98099999999999998},"min":{"1min":0.746,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.722,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.2230000000000001,"15min":1.2390000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.651,"5min":3.286,"15min":3.286},"last":1.302}]},{"osd":5,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.137,"15min":1.165},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5209999999999999,"5min":2.105,"15min":2.105},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.147,"15min":1.169},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.792,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.244}]},{"osd":6,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.25,"15min":1.286},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4590000000000001,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.3520000000000001,"15min":1.373},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6739999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2490000000000001,"15min":1.2529999999999999},"min":{"1min":0.91500000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.8420000000000001,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.387,"15min":1.417},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.5669999999999999,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.4790000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.55400000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0740000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.54500000000000004,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.159,"15min":1.159},"last":1.048}]},{"osd":1,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.745,"15min":0.748},"min":{"1min":0.60599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98299999999999998,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.54400000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.083,"5min":2.012,"15min":2.012},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.64000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.101,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.91100000000000003,"15min":0.90200000000000002},"min":{"1min":0.72999999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3360000000000001,"5min":2.331,"15min":2.331},"last":1.093}]},{"osd":4,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.72899999999999998,"5min":0.624,"15min":0.624},"max":{"1min":1.2969999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.226},{"interface":"front","average":{"1min":1.097,"5min":1.137,"15min":1.143},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.4610000000000001,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.284}]},{"osd":5,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.171,"15min":1.165},"min":{"1min":0.73499999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.169,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.083,"15min":1.073},"min":{"1min":0.93799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.504,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2450000000000001,"15min":1.2569999999999999},"min":{"1min":0.92200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.4690000000000001,"5min":1.946,"15min":1.946},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2809999999999999,"15min":1.2829999999999999},"min":{"1min":0.97399999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.2919999999999998,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.357}]},{"osd":7,"last update":"Fri Oct 6 11:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.175,"15min":1.165},"min":{"1min":0.92500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5780000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3160000000000001,"15min":1.3200000000000001},"min":{"1min":1.0269999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5700000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.4359999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.49299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0620000000000001,"5min":1.236,"15min":1.236},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83999999999999997,"5min":1.171,"15min":1.171},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.45700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.55700000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0880000000000001,"15min":1.091},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1299999999999999,"15min":1.1399999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.238,"5min":1.748,"15min":1.748},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1359999999999999,"15min":1.1319999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.94999999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.1850000000000001},"min":{"1min":0.95399999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.97399999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.522,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.2629999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:23:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.93500000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.224},{"interface":"front","average":{"1min":1.282,"5min":1.25,"15min":1.244},"min":{"1min":1.032,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":828282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.40999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.92600000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94999999999999996,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78800000000000003,"15min":0.78900000000000003},"min":{"1min":0.58499999999999996,"5min":0.314,"15min":0.314},"max":{"1min":1.198,"5min":1.298,"15min":1.298},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89600000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.276,"5min":1.554,"15min":1.554},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.874,"15min":0.878},"min":{"1min":0.64600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0700000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89500000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.159,"5min":1.325,"15min":1.325},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.147,"15min":1.157},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.859,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1599999999999999,"15min":1.1759999999999999},"min":{"1min":0.79300000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3720000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.268},{"interface":"front","average":{"1min":1.123,"5min":1.1599999999999999,"15min":1.1659999999999999},"min":{"1min":0.873,"5min":0.752,"15min":0.752},"max":{"1min":1.4059999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.087}]},{"osd":6,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.153,"15min":1.157},"min":{"1min":0.85499999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5589999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.482},{"interface":"front","average":{"1min":1.157,"5min":1.175,"15min":1.1779999999999999},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.734,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Oct 6 11:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.2529999999999999,"15min":1.2629999999999999},"min":{"1min":0.91300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5249999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.139},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2470000000000001,"15min":1.256},"min":{"1min":0.89300000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4830000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.3819999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":237914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.113}]},{"osd":2,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.111,"15min":1.111},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0640000000000001,"5min":1.331,"15min":1.331},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.77200000000000002,"15min":0.78400000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95499999999999996,"5min":1.091,"15min":1.091},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97699999999999998,"15min":0.98099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3260000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.155},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.419,"5min":1.534,"15min":1.534},"last":0.98699999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.097,"15min":1.095},"min":{"1min":0.92100000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.3140000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0429999999999999,"15min":1.0309999999999999},"min":{"1min":0.82799999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.4910000000000001,"5min":1.613,"15min":1.613},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.131,"5min":1.1679999999999999,"15min":1.1739999999999999},"min":{"1min":0.92500000000000004,"5min":0.752,"15min":0.752},"max":{"1min":1.393,"5min":1.583,"15min":1.583},"last":1.25}]},{"osd":7,"last update":"Fri Oct 6 11:23:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.94599999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4410000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.331},{"interface":"front","average":{"1min":1.181,"5min":1.149,"15min":1.143},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.4259999999999999,"5min":1.504,"15min":1.504},"last":1.0640000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:24:05.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T11:24:05.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:05.245 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 Processor -- start 2023-10-06T11:24:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 -- start start 2023-10-06T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4100aaee0 0x7fe4100ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4100cc1d0 0x7fe4100b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100ac8e0 con 0x7fe4100ae810 2023-10-06T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe410156110 con 0x7fe4100aaee0 2023-10-06T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe416c28700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ab2b0 con 0x7fe4100cc1d0 2023-10-06T11:24:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4100aaee0 0x7fe4100ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4100aaee0 0x7fe4100ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50538/0 (socket says 172.21.15.7:50538) 2023-10-06T11:24:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 -- 172.21.15.7:0/1335901098 learned_addr learned my addr 172.21.15.7:0/1335901098 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:24:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe40ffff700 1 --2- 172.21.15.7:0/1335901098 >> v2:172.21.15.50:3300/0 conn(0x7fe4100cc1d0 0x7fe4100b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 -- 172.21.15.7:0/1335901098 >> v2:172.21.15.50:3300/0 conn(0x7fe4100cc1d0 msgr2=0x7fe4100b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 --2- 172.21.15.7:0/1335901098 >> v2:172.21.15.50:3300/0 conn(0x7fe4100cc1d0 0x7fe4100b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 -- 172.21.15.7:0/1335901098 >> v1:172.21.15.50:6789/0 conn(0x7fe4100ae810 legacy=0x7fe4100c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.346+0000 7fe4149c4700 1 -- 172.21.15.7:0/1335901098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100acbc0 con 0x7fe4100aaee0 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe4149c4700 1 --2- 172.21.15.7:0/1335901098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100aaee0 0x7fe4100ad980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe410068f10 tx=0x7fe404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8bf60fab6168a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe40f7fe700 1 -- 172.21.15.7:0/1335901098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404009e70 con 0x7fe4100aaee0 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe40f7fe700 1 -- 172.21.15.7:0/1335901098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40400b860 con 0x7fe4100aaee0 2023-10-06T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe40f7fe700 1 -- 172.21.15.7:0/1335901098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40400bc00 con 0x7fe4100aaee0 2023-10-06T11:24:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- 172.21.15.7:0/1335901098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100aaee0 msgr2=0x7fe4100ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 --2- 172.21.15.7:0/1335901098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100aaee0 0x7fe4100ad980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe410068f10 tx=0x7fe404009250 comp rx=0 tx=0).stop 2023-10-06T11:24:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- 172.21.15.7:0/1335901098 shutdown_connections 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 --2- 172.21.15.7:0/1335901098 >> v2:172.21.15.50:3300/0 conn(0x7fe4100cc1d0 0x7fe4100b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 --2- 172.21.15.7:0/1335901098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100aaee0 0x7fe4100ad980 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- 172.21.15.7:0/1335901098 >> 172.21.15.7:0/1335901098 conn(0x7fe410068900 msgr2=0x7fe4100cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- 172.21.15.7:0/1335901098 shutdown_connections 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- 172.21.15.7:0/1335901098 wait complete. 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 Processor -- start 2023-10-06T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.350+0000 7fe416c28700 1 -- start start 2023-10-06T11:24:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4100aaee0 0x7fe41014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe4100cc1d0 0x7fe4101534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410153d50 con 0x7fe4100ae810 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410154030 con 0x7fe4100aaee0 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe410154310 con 0x7fe4100cc1d0 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50554/0 (socket says 172.21.15.7:50554) 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 -- 172.21.15.7:0/2664469111 learned_addr learned my addr 172.21.15.7:0/2664469111 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe4149c4700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4100aaee0 0x7fe41014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe4151c5700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe4100cc1d0 0x7fe4101534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 -- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe4100cc1d0 msgr2=0x7fe4101534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe4100cc1d0 0x7fe4101534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 -- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4100aaee0 msgr2=0x7fe41014f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4100aaee0 0x7fe41014f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 -- 172.21.15.7:0/2664469111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040094f0 con 0x7fe4100ae810 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40ffff700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e950 tx=0x7fe3fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc013680 con 0x7fe4100ae810 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0137e0 con 0x7fe4100ae810 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4101545f0 con 0x7fe4100ae810 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc013ba0 con 0x7fe4100ae810 2023-10-06T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.354+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe410153a30 con 0x7fe4100ae810 2023-10-06T11:24:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe3fc0123d0 con 0x7fe4100ae810 2023-10-06T11:24:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41004be00 con 0x7fe4100ae810 2023-10-06T11:24:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe40cff9700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 0x7fe3f4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fe3fc07cc70 con 0x7fe4100ae810 2023-10-06T11:24:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe4149c4700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 0x7fe3f4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.358+0000 7fe4149c4700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 0x7fe3f4062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe410068f10 tx=0x7fe404017000 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.366+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe3fc04cee0 con 0x7fe4100ae810 2023-10-06T11:24:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.522+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe4100c8640 con 0x7fe4100ae810 2023-10-06T11:24:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.530+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe3fc04cee0 con 0x7fe4100ae810 2023-10-06T11:24:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.530+0000 7fe40cff9700 1 -- 172.21.15.7:0/2664469111 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe3fc050500 con 0x7fe4100ae810 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 msgr2=0x7fe3f4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 0x7fe3f4062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe410068f10 tx=0x7fe404017000 comp rx=0 tx=0).stop 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 msgr2=0x7fe410152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e950 tx=0x7fe3fc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 shutdown_connections 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7fe3f4060670 0x7fe3f4062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7fe4100cc1d0 0x7fe4101534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4100ae810 0x7fe410152fb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 --2- 172.21.15.7:0/2664469111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4100aaee0 0x7fe41014f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 >> 172.21.15.7:0/2664469111 conn(0x7fe410068900 msgr2=0x7fe4100c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 shutdown_connections 2023-10-06T11:24:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.534+0000 7fe416c28700 1 -- 172.21.15.7:0/2664469111 wait complete. 2023-10-06T11:24:05.550 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:24:05.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:24:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.646+0000 7ff692570700 1 Processor -- start 2023-10-06T11:24:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.646+0000 7ff692570700 1 -- start start 2023-10-06T11:24:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff68c15c540 0x7ff68c0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff68c15c980 0x7ff68c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c0b04f0 con 0x7ff68c15c170 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c0af8e0 con 0x7ff68c15c540 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c0afbc0 con 0x7ff68c15c980 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff68c15c540 0x7ff68c0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff68c15c540 0x7ff68c0af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50560/0 (socket says 172.21.15.7:50560) 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 -- 172.21.15.7:0/1908518553 learned_addr learned my addr 172.21.15.7:0/1908518553 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff691d6f700 1 --2- 172.21.15.7:0/1908518553 >> v2:172.21.15.50:3300/0 conn(0x7ff68c15c980 0x7ff68c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 -- 172.21.15.7:0/1908518553 >> v2:172.21.15.50:3300/0 conn(0x7ff68c15c980 msgr2=0x7ff68c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 --2- 172.21.15.7:0/1908518553 >> v2:172.21.15.50:3300/0 conn(0x7ff68c15c980 0x7ff68c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 -- 172.21.15.7:0/1908518553 >> v1:172.21.15.50:6789/0 conn(0x7ff68c15c170 legacy=0x7ff68c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 -- 172.21.15.7:0/1908518553 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0afea0 con 0x7ff68c15c540 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff690d6d700 1 --2- 172.21.15.7:0/1908518553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c540 0x7ff68c0af2d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0b4830 tx=0x7ff680009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7c010f266503e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff67ffff700 1 -- 172.21.15.7:0/1908518553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680009e70 con 0x7ff68c15c540 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff67ffff700 1 -- 172.21.15.7:0/1908518553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff68000b860 con 0x7ff68c15c540 2023-10-06T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff67ffff700 1 -- 172.21.15.7:0/1908518553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000bc20 con 0x7ff68c15c540 2023-10-06T11:24:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- 172.21.15.7:0/1908518553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c540 msgr2=0x7ff68c0af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 --2- 172.21.15.7:0/1908518553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c540 0x7ff68c0af2d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0b4830 tx=0x7ff680009250 comp rx=0 tx=0).stop 2023-10-06T11:24:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- 172.21.15.7:0/1908518553 shutdown_connections 2023-10-06T11:24:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 --2- 172.21.15.7:0/1908518553 >> v2:172.21.15.50:3300/0 conn(0x7ff68c15c980 0x7ff68c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 --2- 172.21.15.7:0/1908518553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c540 0x7ff68c0af2d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- 172.21.15.7:0/1908518553 >> 172.21.15.7:0/1908518553 conn(0x7ff68c068900 msgr2=0x7ff68c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- 172.21.15.7:0/1908518553 shutdown_connections 2023-10-06T11:24:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 -- 172.21.15.7:0/1908518553 wait complete. 2023-10-06T11:24:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.650+0000 7ff692570700 1 Processor -- start 2023-10-06T11:24:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- start start 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 0x7ff68c0a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 --2- >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff68c15c540 0x7ff68c0a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ae1b0 con 0x7ff68c15c980 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ad5a0 con 0x7ff68c15c170 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- --> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ad880 con 0x7ff68c15c540 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50564/0 (socket says 172.21.15.7:50564) 2023-10-06T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 -- 172.21.15.7:0/930674220 learned_addr learned my addr 172.21.15.7:0/930674220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-06T11:24:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff69156e700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 0x7ff68c0a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff690d6d700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff68c15c540 0x7ff68c0a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 -- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff68c15c540 msgr2=0x7ff68c0a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff68c15c540 0x7ff68c0a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 -- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 msgr2=0x7ff68c0a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 0x7ff68c0a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 -- 172.21.15.7:0/930674220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094f0 con 0x7ff68c15c980 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff69156e700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 0x7ff68c0a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff691d6f700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff684009e90 tx=0x7ff68400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68400cb70 con 0x7ff68c15c980 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff68400f040 con 0x7ff68c15c980 2023-10-06T11:24:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c0adb60 con 0x7ff68c15c980 2023-10-06T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684014480 con 0x7ff68c15c980 2023-10-06T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.654+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c0ae640 con 0x7ff68c15c980 2023-10-06T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c0ae320 con 0x7ff68c15c980 2023-10-06T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff684014830 con 0x7ff68c15c980 2023-10-06T11:24:05.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff67d7fa700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 0x7ff670062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:24:05.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff68407d610 con 0x7ff68c15c980 2023-10-06T11:24:05.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff69156e700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 0x7ff670062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:24:05.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.658+0000 7ff69156e700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 0x7ff670062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff688001d20 tx=0x7ff688009040 comp rx=0 tx=0).ready entity=mgr.5458 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:24:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.666+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff68404d890 con 0x7ff68c15c980 2023-10-06T11:24:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.862+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68c0b0180 con 0x7ff68c15c980 2023-10-06T11:24:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.862+0000 7ff67d7fa700 1 -- 172.21.15.7:0/930674220 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3514 (secure 0 0 0) 0x7ff684018dd0 con 0x7ff68c15c980 2023-10-06T11:24:05.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-06T11:24:05.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/177371564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":177371564},{"type":"v1","addr":"172.21.15.50:6835","nonce":177371564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/469634277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":469634277},{"type":"v1","addr":"172.21.15.50:6837","nonce":469634277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5199,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3310075031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3310075031},{"type":"v1","addr":"172.21.15.7:6833","nonce":3310075031}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/654536126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":654536126},{"type":"v1","addr":"172.21.15.7:6835","nonce":654536126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-06T11:24:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 msgr2=0x7ff670062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 0x7ff670062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff688001d20 tx=0x7ff688009040 comp rx=0 tx=0).stop 2023-10-06T11:24:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 msgr2=0x7ff68c0cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff684009e90 tx=0x7ff68400b040 comp rx=0 tx=0).stop 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 shutdown_connections 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:6832/33317,v1:172.21.15.50:6833/33317] conn(0x7ff670060670 0x7ff670062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68c15c980 0x7ff68c0cad70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3301/0,v1:172.21.15.50:6790/0] conn(0x7ff68c15c540 0x7ff68c0a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 --2- 172.21.15.7:0/930674220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff68c15c170 0x7ff68c0a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 >> 172.21.15.7:0/930674220 conn(0x7ff68c068900 msgr2=0x7ff68c0cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 shutdown_connections 2023-10-06T11:24:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-06T11:24:05.866+0000 7ff692570700 1 -- 172.21.15.7:0/930674220 wait complete. 2023-10-06T11:24:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2023-10-06T11:24:05.883 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T11:24:05.883 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T11:24:05.884 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:24:05.884 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:05.899 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:24:05.899 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:24:05.899 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:05.910 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:24:05.911 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:24:05.911 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:05.928 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:24:05.928 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:24:05.928 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:05.945 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:24:05.945 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T11:24:05.946 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T11:24:05.946 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.026 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T11:24:06.026 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T11:24:06.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.100 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T11:24:06.101 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T11:24:06.101 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.166 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T11:24:06.166 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T11:24:06.166 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.264 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T11:24:06.264 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T11:24:06.264 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.342 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T11:24:06.342 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T11:24:06.342 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.416 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T11:24:06.417 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T11:24:06.417 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.479 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T11:24:06.479 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T11:24:06.479 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.542 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T11:24:06.542 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T11:24:06.542 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T11:24:06.542 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.570 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T11:24:06.571 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T11:24:06.571 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.597 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T11:24:06.597 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T11:24:06.598 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T11:24:06.598 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.617 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T11:24:06.618 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T11:24:06.618 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.660 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T11:24:06.660 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T11:24:06.661 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:24:06.678 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T11:24:06.678 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T11:24:06.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T11:24:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:1696591312.1584105 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T11:24:06.701 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T11:24:06.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T11:24:06.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T11:24:06.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T11:24:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:1696591312.1584105 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T11:24:06.795 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:24:06.795 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T11:24:07.136 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:24:07.136 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T11:24:07.389 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:24:07.390 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T11:24:07.547 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:24:07.547 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T11:24:07.698 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:24:07.698 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T11:24:07.967 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:24:07.967 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T11:24:08.108 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:24:08.109 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T11:24:08.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:24:08.234 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T11:24:08.375 INFO:tasks.ceph:Archiving mon data... 2023-10-06T11:24:08.377 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/data/mon.a.tgz 2023-10-06T11:24:08.377 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-10-06T11:24:08.384 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.3N6pClCHuR 2023-10-06T11:24:08.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3N6pClCHuR 2023-10-06T11:24:08.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.3N6pClCHuR 2023-10-06T11:24:08.533 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.3N6pClCHuR is 606KB 2023-10-06T11:24:08.599 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.3N6pClCHuR 2023-10-06T11:24:08.606 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/data/mon.b.tgz 2023-10-06T11:24:08.606 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-10-06T11:24:08.613 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.ZsnTU4EfP2 2023-10-06T11:24:08.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ZsnTU4EfP2 2023-10-06T11:24:08.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.ZsnTU4EfP2 2023-10-06T11:24:08.848 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.ZsnTU4EfP2 is 859KB 2023-10-06T11:24:08.881 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.ZsnTU4EfP2 2023-10-06T11:24:08.887 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/data/mon.c.tgz 2023-10-06T11:24:08.887 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-10-06T11:24:08.935 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.uzxDIUiycs 2023-10-06T11:24:08.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uzxDIUiycs 2023-10-06T11:24:09.099 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.uzxDIUiycs 2023-10-06T11:24:09.116 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.uzxDIUiycs is 859KB 2023-10-06T11:24:09.184 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.uzxDIUiycs 2023-10-06T11:24:09.191 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T11:24:09.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T11:24:09.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T11:24:09.209 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T11:24:09.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T11:24:09.250 DEBUG:teuthology.orchestra.run.smithi050:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T11:24:09.270 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T11:24:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T11:24:09.282 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T11:24:09.284 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi007/crash 2023-10-06T11:24:09.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T11:24:09.302 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi050/crash 2023-10-06T11:24:09.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T11:24:09.342 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T11:24:09.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:09.345 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:09.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:09.441 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:09.457 INFO:tasks.ceph:Compressing logs... 2023-10-06T11:24:09.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:24:09.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:24:13.148 INFO:tasks.ceph:Archiving logs... 2023-10-06T11:24:13.148 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi007/log 2023-10-06T11:24:13.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T11:24:13.738 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi050/log 2023-10-06T11:24:13.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T11:24:14.147 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T11:24:14.156 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T11:24:14.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T11:24:14.158 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T11:24:14.238 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T11:24:14.247 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T11:24:14.247 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:24:14.249 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:24:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:24:14.273 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2023-10-06T11:24:14.273 INFO:teuthology.orchestra.run.smithi007.stdout:*hv01.front.sepi 44.4.53.2 2 u 29 64 77 0.071 2.630 1.455 2023-10-06T11:24:14.273 INFO:teuthology.orchestra.run.smithi007.stdout:+hv02.front.sepi 63.231.80.2 3 u 32 64 77 0.104 0.064 2.060 2023-10-06T11:24:14.273 INFO:teuthology.orchestra.run.smithi007.stdout:+hv03.front.sepi 74.6.168.72 3 u 30 64 77 0.073 1.653 1.862 2023-10-06T11:24:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout:*hv01.front.sepi 44.4.53.2 2 u 29 64 77 0.088 1.476 0.999 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout:+hv02.front.sepi 63.231.80.2 3 u 27 64 77 0.092 -0.129 0.930 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout:+hv03.front.sepi 74.6.168.72 3 u 25 64 77 0.103 -1.457 1.068 2023-10-06T11:24:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:24:14.276 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T11:24:14.285 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T11:24:14.286 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T11:24:14.294 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T11:24:14.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T11:24:14.375 INFO:teuthology.task.internal:Duration was 710.816176 seconds 2023-10-06T11:24:14.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T11:24:14.398 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T11:24:14.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T11:24:14.401 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T11:24:14.452 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T11:24:14.452 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2023-10-06T11:24:14.453 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T11:24:14.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2023-10-06T11:24:14.464 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T11:24:14.476 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T11:24:14.477 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:24:14.479 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:24:14.509 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T11:24:14.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T11:24:14.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T11:24:14.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T11:24:14.627 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T11:24:14.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T11:24:14.645 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T11:24:14.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T11:24:14.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:24:14.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:24:14.688 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2023-10-06T11:24:14.688 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2023-10-06T11:24:14.705 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:24:14.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:24:14.750 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:24:14.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:24:14.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T11:24:14.766 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T11:24:14.767 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi007 2023-10-06T11:24:14.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T11:24:14.789 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395/remote/smithi050 2023-10-06T11:24:14.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T11:24:14.816 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T11:24:14.816 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T11:24:14.819 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T11:24:14.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T11:24:14.836 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T11:24:14.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T11:24:14.862 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T11:24:14.863 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T11:24:14.866 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T11:24:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 11:24 /home/ubuntu/cephtest 2023-10-06T11:24:14.877 INFO:teuthology.orchestra.run.smithi050.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 11:24 /home/ubuntu/cephtest 2023-10-06T11:24:14.879 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T11:24:14.890 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T11:24:14.928 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T11:24:14.953 DEBUG:teuthology.nuke:shortname: smithi007 2023-10-06T11:24:14.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:24:14.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:03:08.080798', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T11:24:14.985 DEBUG:teuthology.nuke:shortname: smithi050 2023-10-06T11:24:14.985 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:24:15.008 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:03:08.082571', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T11:24:15.035 INFO:teuthology.orchestra.console.smithi007:Power off 2023-10-06T11:24:15.036 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:24:15.058 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2023-10-06T11:24:15.058 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:15.078 INFO:teuthology.orchestra.console.smithi050:Power off 2023-10-06T11:24:15.078 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:24:15.088 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-10-06T11:24:15.105 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2023-10-06T11:24:15.106 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:15.129 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-10-06T11:24:19.089 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:19.130 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:19.199 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-10-06T11:24:19.242 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-10-06T11:24:23.202 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:23.243 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:24:23.312 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2023-10-06T11:24:23.353 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-10-06T11:24:23.413 INFO:teuthology.orchestra.console.smithi007:Power off completed 2023-10-06T11:24:23.454 INFO:teuthology.orchestra.console.smithi050:Power off completed 2023-10-06T11:24:23.560 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} duration: 710.8161761760712 failure_reason: '"1696591312.1584105 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T11:24:23.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:24:23.638 INFO:teuthology.run:FAIL